SuperTECH NEWS is the bi-monthly newsletter of the BLE GROUP, which provides small- and medium-size school systems with supplementary technology management to produce high-quality educational results and efficient management.

The purpose of SuperTECH NEWS is to provide education decision makers with concise information that allows them to make informed technology decisions to impact instruction, management and communication. This is information you can use on Monday morning.

Editor, Susan DeMark
Web Design, Charlene Polanosky
Publisher, Eliot Levinson

Our February issue theme is Web-based assessment products for high-stakes tests. Choose from the following articles.

	Note from Eliot—Launching the new No Child Left Behind (NCLB) Assessment and Management Service. The BLE Group is launching a new service for small and medium-size school systems. We assess where school districts are currently and exactly what they have to do to implement NCLB. The service also provides school systems the supplementary management support they need to purchase and implement the new technology-delivered programs.
	Theme of the Month—Online high-stakes testing. Online testing is the first line of products many school systems will buy to address NCLB. These products pinpoint student performance, deliver test results instantaneously, and inform instruction for every student. It is not simple to purchase these new products. This section gives you information on how to intelligently purchase and implement these products.
	Products—Within the next two years, many school districts will adopt the use of online assessment products aligned with state standards and high-stakes state tests. We provide you with in-depth, concise, and vendor-neutral descriptions of a number of leading Web-based and Web-enabled assessment products.
	Best Practices —Noni Miller, who heads educational services in Ypsilanti, Michigan, Public Schools, shares lessons on how her district is using a leading online standards-based assessment program to drive instruction throughout the school system and to improve students' reading and math proficiency.
	Conferences—Check out the relevant conferences coming in the next several months.

We want to hear from you. What do you agree and disagree with on this issue (we will post comments from readers in the next issue). Please write us at eliot@blegroup.com.

DID YOU MISS AN ISSUE?
You can read past issues of SuperTECH NEWS online:

• December 2002 - Focus: Student Information Systems
• August 2002 - Focus: Purchasing Hardware 2002
• June 2002 - Focus: Web-based Applications for Early Reading
• May 2002 - Focus: Web-based Professional Development
• March 2002 - Focus: Technology of Accountability

THE BLE Group's NCLB Assessment and Management Service

Why are we launching the NCLB Assessment and Management Service?

Technology is necessary to implement the No Child Left Behind Act. Technology is a central component to the solution of every facet of NCLB, whether it is teacher quality, the delivery of standards-based instruction, assessment, the monitoring of student progress, school-parent communication, or reading proficiency.

The BLE Group knows that small- and medium-size school systems—the 86 percent of school districts in the U.S. with fewer than 5,000 students—often lack the extensive resources and knowledge base to implement NCLB. They lack sufficient technologists and technology-savvy educators to plan and manage the Web-based instructional programs and assessment systems that are the solutions for NCLB. We make available reasonably priced services that can provide the expertise school systems need to address NCLB on a time-shared basis.

Excellent technology staff is expensive and hard to find. We've created the NCLB Assessment and Management Service as a means of supplementing the staffs of small school systems with our own team of skilled technologists and technology-savvy educators. We will help you plan and execute an effective NCLB program.

The NCLB Assessment and Management Service supplements the instructional and evaluation capability of small and mid-sized school systems so that they can effectively address No Child Left Behind. There are two tiers to the NCLB Assessment and Management Service. Tier 1—the NCLB assessment and plan—creates an assessment for districts on how effectively they are currently addressing the multiple requirements of NCLB, such as teacher quality, assessment, reading achievement, etc., and devises a specific plan to address NCLB. The plan includes new technology-based solutions, a schedule, and a timeline for addressing NCLB. Tier 2—the management service—supplies ongoing management support to districts. We furnish districts with ongoing service from the BLE Group to purchase products and provide supplementary management as districts implement their NCLB programs.

The NCLB Assessment and Management Service evaluates how well are you currently addressing NCLB and delivering on its mandates, and it centers on exactly what you should do over the next year to implement NCLB effectively so that your district's performance improves.

Why are the specific areas of the BLE Group's NCLB assessment and plan?

The BLE Group provides an assessment and solution for the following NCLB requirements:

• Reading—Includes benchmarks, diagnostic testing
• Teacher Quality—Certification, paraprofessional certification, online training
• Testing—State standards, diagnostic testing
• Staff development—What is needed to meet certification, improve standards-based teaching, address technology skills linked to teaching
• Paraprofessionals—Tracking certification
• Management of NCLB—Planning for low-performing schools
• Information Analysis—The know-how to aggregate and disaggregate scores
• Grant proposals—What information is needed for the annual district proposal to include all students
• State accountability—What does the state have to do to improve accountability
  

The NCLB assessment and plan focuses on instruction, assessment, management systems, and technology. It includes:

• An annual implementation plan. Quarter by quarter, the plan lays out what has to be done in each of the 4 areas described above.
• A budget.
• An assessment of the current state of NCLB linked with specific recommendations on the items listed above.
  

NCLB Supplementary Management Service

Following are the supplementary management services that districts can make use of after the BLE Group assessment and plan. BLE Group CIOs and curriculum directors will supplement the district's staff with the following services:

• RFPs. For strategic systems purchases such as instructional management and on line assessment systems.
• Review of contracts. BLE will review district technology contracts and write effective contracts for the district.
• Monthly phone consultations and quarterly visits to address NCLB management.
• Vendor Management. BLE Group will oversee your NCLB vendors.
• Access to databases on instructional and administrative systems. BLE Group maintains confidential databases on management and instructional software for its' clients.
• Discounts from collaborative buying of hardware and instructional, assessment, and management software.
• SuperTECH NEWS newsletter. The newsletter delivers information to administrators on NCLB-related technology issues such as assessment, data warehousing, and instructional management.
  

If you are interested in the NCLB Assessment and Management Service, please contact us to discuss the matter further. The cost is reasonable.

Eliot Levinson <eliot@blegroup.com>, CEO,
THE BLE GROUP
202.281.1763

Levings Learning, based in Oklahoma City, has developed Web-based assessments for grades 3-12 in mathematics, English, social science, science, and fine arts. The company has designed the assessments to be fully aligned with state standards in 49 states (Iowa does not have separate state standards). The company's package combines assessment, evaluation, and real-time results. With the company's PASS Plan, teachers, parents, and administrators can determine how students are progressing using real-time data; assess whether students understand and grasp the concepts required by standards; and gauge whether the questions really align to standards and assess an individual concept.

The Levings Learning Internet-based program gives districts the ability to create their own local assessments benchmarked to state standards. Teachers can dynamically develop assessments for their classes, using the company's pool of assessment items. Levings Learning's tools also let districts import their own tests into the program for use. The assessments can be administered in a variety of ways: taken online on regular computers; using handheld computers; or taken off-line, scored with a scanner, and imported back into the online system. Teachers can score them using online rubrics or they may generate their own rubrics. The reporting tool built into the assessment program permits teachers and administrators to aggregate and disaggregate the data by varied categories.

Levings Learning has districts in 19 states using the program, and is aiming to be in all 50 states by mid-year.

The company's in-house team contains a large number of teachers and curriculum developers, and an external group of consultants, who include teachers and curriculum experts, continually adds items to the assessment pool. Also, the company has licensed approximately one-third of its items from publishers and other sources, such as the Colorado-based Center for Performance Assessment. Currently, about 25 percent of the items in the program are ones developed by Levings, and 75 percent are items added in by districts and states. There are approximately 26,000 items currently in the item database.

To assure reliability of its assessments, the company has two strategies: It depends on the credibility and reliability of the publishers with whom it partners. Also, the company continually analyzes how items are performing relative to state standards and revises items, based on feedback from the districts and its analysis.

The cost of Levings Learning program is on a per-student or per-site basis. The price is all-inclusive, and there are no additional costs for added modules.

Lightspan eduTest is a customizable, online benchmark assessment program that assesses students relative to state and national standards. The company's Assessment Builder offers a template-based tool that guides teachers and educators in designing assessments specific to district needs and state objectives. End users can address the standards, assessments, and test parameters that the district has established, and the program permits them to have control over the length, subject matter, and content of assessments. Essentially, the Lightspan products couple test content with a tool that allows customized test creation, and everything is Web-based with passwords and secure technology.

EduTest provides fixed benchmark tests that can be administered multiple times during a school year, at the beginning, middle, and end of the year. Teachers can obtain real-time student performance data throughout the school year that enables them to target instruction. Comprehensive reports allow them to identify the needs of each student by specific standards, strands, and objectives. Administrators can also access the data at the district, school, and classroom level. How it works: Once a district purchases the program, those in the district create assessments using the Lightspan Assessment Builder; deliver assessments through the eduTest assessment; and receive the results. Teachers and students obtain feedback immediately, and district administrators receive district and school-aggregated scores within hours. Reports can be generated and provided to parents in order to see how their child is progressing against various objectives and standards.

Using the Lightspan assessment, one can compare assessment results to high-stakes tests and previous performance reports, and analyze your district's data by gender, age, socioeconomic status, and other criteria to comply with No Child Left Behind. Student data can be disaggregated by these varied categories, including gender, Title I, etc. Districts can upload and disaggregate the data through the district's Student Information System.

The assessments are criterion-referenced tests that report how well students are performing relative to pre-determined levels on specific goals or objectives. EduTest assessments cover grades 2-8 in math, reading, and language arts.

Users can access a search engine with more than 60,000 test items, and they may also develop their own test items for use. Currently, the items can address state and national standards. The item pool covers seven states with complete alignment; Lightspan plans to launch complete coverage for all states with state standards in the near future. The content writers for Lightspan have educational backgrounds, and include teachers, content specialists, and administrators.

Lightspan employs internal and external measures of reliability and validity. All Lightspan tests and test items are reviewed and developed following accepted standards for low-stakes, criterion-referenced tests. A panel of educators and assessment experts are employed to review and validate Lightspan's test items. As a measure of test validity, correlation studies have been conducted to examine the relationship between performance on specific Lightspan eduTest assessment tests and state-mandated criterion-referenced tests in multiple states.

To examine test reliability, internal consistency estimates of eduTest assessment test instruments have been calculated based on test results from samples of students in various states. In reliability studies, high internal consistency values were observed for the Lightspan eduTest assessment benchmark tests evaluated. In addition, Lightspan conducts alignment studies to determine the validity of its assessments with state standards. The company recently announced a partnership was with WestEd, a California nonprofit education research agency, to review a subset of Lightspan's assessment content.

The Lightspan assessment products are sold by subscription by the school, district, or state levels. Prices vary considerably depending on the way it is purchased and what users choose as part of the assessment suite.

The Northwest Evaluation Association (NWEA), a Portland, Oregon-based nonprofit organization, has created and offers a computerized adaptive test known as Measures of Academic Progress (MAP). The assessment aims to meet the goals of the No Child Left Behind Act by testing and measuring a student's academic growth in specific subjects over time. MAP is electronically delivered and is Web-enabled.

The NWEA assessments cover grades 2-10 in reading, math, and language arts. They currently are aligned to standards in 48 states. The organization customizes tests in math, reading, and language usage using the NWEA item pools to create tests that align with local curriculum and state standards. NWEA works with districts across the country, and also has the capability of working with large state contracts. For example, NWEA is the primary contractor on a statewide Internet-based adaptive assessment in Idaho for students in grades 2-9. The organization also currently has partnerships with 716 school districts.

While the student is taking the computerized adaptive test, it adjusts and customizes to the student's achievement level. NWEA delivers the assessment to a district through a Web-connected machine and it is then distributed to students' computers that are connected to the district's server. Once the assessment is completed, the MAP data is uploaded to NWEA through the Web-enabled computer and processed. The day after the assessment, teachers and schools have access to the performance results online.

Teachers receive a special report of the test results, and a suite of reports is available on the assessment that can be aggregated or disaggregated, according to various filters, including Special Education students, Title I students, gender, etc.

NWEA develops all of the items for its MAP, and train item writers, utilizing the services of teachers from all over the country. Submitted items go through a content and bias review, and then they are field-tested with hundreds of students. Once they have gone through this process and are approved, they are placed in the item pool. NWEA currently has 25,000 unique items in its item pool.

A research team from NWEA monitors the assessments and conducts validity and reliability tests. NWEA recently completed research concerning the stability of its measurement scale, and found that it has been stable for the past 20 years. The measurement scale crosses grade levels, and it is a continuous scale across all levels. This allows NWEA to track individual growth, in other words to compare individuals to themselves over time. If you look at how a student is performing with regard to the scale throughout the year, educators can predict how a student will do on a state test later in the year. Because thousands of students take NWEA's assessments, the organization has the ability to report normative data, too, a company official said.

Training for the staff begins with initial face-to-face meetings that explain how a test system will work within a district's plans and regarding educational goals, prior to any setup of the assessments. Prior to the administration of assessments, teachers and administrators receive training in the technology, in the setup of assessments, in administration procedures. Once assessments are given and reports are first made, the group provides training in interpreting the data to make maximum use of the assessments to inform instruction. After two seasons of testing, NEWA does another round of training, focusing on how to interpret a student's academic growth. Training is done on-site, working with staff representatives who in turn train others.

The cost of the NWEA computer adaptive assessments is based on enrollment. Most common is a district license. NWEA, for instance, computes the total enrollment of a district in K-12, and multiplies it by $5.25 per student, to arrive at the fee. For this amount, the district gets tests in reading, math, and language arts, and can test students in grades 2-10 up to four times a year. This amount does not include training. The training is in addition to the basic cost and is sold by module.

Through Homeroom.com, Princeton Review offers an ongoing and online formative assessment and benchmarking tool aligned to state standards and high-stakes state tests. Teachers can create customized practice assessments, using the Homeroom.com pool of thousands of items, for grades 3-12, and assess and diagnose students' performance specific to the skills measured by a state's tests. The product is completely Web-based and delivered through a Web browser. It was launched in 1999. Homeroom is sold on a statewide as well as district-wide basis. The company has a statewide contract with Massachusetts, and district wide contracts across the country including Philadelphia and Memphis, to name a few.

The math and reading content of Homeroom.com is aligned to multi-state and state tests, all state standards, and state textbooks. Students can take the tests either online or off-line. In real time, Homeroom.com provides skills-based assessment results at the class and student level, and the results can be drilled down to objective and item. The up-to-the-minute results are available to students, teachers, parents, principals, and district administrators, and allow the tracking of students' strengths and weaknesses on state-mandated skills, according to the guidelines of No Child Left Behind.

Homeroom.com assessment segment is complemented by a remediation component through targeted online resources. Homeroom.com makes available targeted skills practice with activities, lessons, and tips. The product has more than 10,000 resources, ranging from games to academic worksheets, each geared to work with individual learning styles. Using Homeroom.com, for instance, parents can not only monitor skill performance and see a child's latest assignments, but also find resources that are intended to help children improve their skills.

Homeroom.com content currently covers nearly all of the 50 states in the U.S. Such coverage varies from some states in which Homeroom.com has the exact idiom and format of a state test to other states in which the content basically matches the skills covered in the state high-stakes assessment.

In-depth reports allow administrators to access up-to-the-minute results at the district, school, class, grade, and student levels. The reporting capability of Homeroom.com lets school administrators and teachers disaggregate the data by varied parameters according to NCLB mandates, such as gender. The reports available to teachers include the overall performance summary chart; student performance summary; topic and skill analysis by student; and item analysis by student. The data auto-import services of Homeroom.com allow student, teacher, and administrator data to be imported into Homeroom automatically so that teachers and administrators do not have to spend time setting up each student's account.

In creating an item pool, Princeton Review is actively involved in understanding and reviewing all kinds of students assessments, and applies that knowledge to item-development efforts to present students with a wide variety of items to look like, feel like, and test the same content as any norm- or criterion- referenced test. Test items are written by The Princeton Review's staff of educators based on analysis of all standards and tests. There are 160,000 items in the item pool.

As items are created, content editors, text editors, and copyeditors review them for accuracy and to ensure that they are error-free. Content reviewers also evaluate the questions and passages for any possible bias. Once items have been through several reviews, they are ready for inclusion in the item bank database.

Each item is associated with skills that serve as the backbone of the Homeroom system, allowing users to cross-reference their curricular goals or standards with their testing requirements. Items are not initially field-tested. Instead, they immediately become available to users. Princeton Review makes some assumptions about the quality of the items because the company considers the source items to be reliable, valid, and, at the very least, of significant difficulty.

Princeton Review, however, does not rely solely on such assumptions to guarantee a valid and reliable item bank. The items are evaluated using traditional item statistics. Items are modified and/or removed if they are determined to be too difficult, or if a single distracter is impacting student performance. As the assessment product develops, the company is continually looking at ways to improve our item bank, a company official said. While the company does not perform calibration studies or employ item response theory, Princeton review believes that the item bank is strong and able to guide instruction when used appropriately.

Access to Homeroom.com is sold on a subscription basis. The cost is based on the number of students within the state/district/school who will be using the product.

Renaissance Learning offers StandardsMaster, an instant assessment coupled with Web-based reporting software that covers math, reading, and language arts in grades 3-10. StandardsMaster software connects the district office and schools in a network that makes state standards assessment data available to teachers, students, parents, principals, and district administrators. StandardsMaster was launched on April 1, 2002, and last September, the company added online testing capability.

With the StandardsMaster program, schools have the capability to administer either a paper-and-pencil or an online test - in math, reading, and language arts - that monitors every student's progress toward state standards. The company is developing content aligned to all 50 states, and has produced content aligned to about half of those states thus far. The company expects to complete content aligned for all states by the next school year, 2003-2004, a company official said. Through the criterion-referenced assessments, educators can analyze specific areas in which students need aid in mastering state standards.

The assessments are done in several steps: Teachers can print out an assessment, using the Web-based software, and administer it to students using the paper-and-pencil method, similar to most standardized tests. Using the AccelScan Intelligent Mark Recognition Reader, teachers score the assessments in their own classrooms, a process that can be done within minutes of the test completion. Once the bubble card from the test is fed through a scanner, you can access the test results from any Web-connected computer in the network.

Customizable reports on the test results can be created at the district, school, classroom, and student level. Reports can be disaggregated using filters such as gender, ESL students, Title I students, etc. Teachers and students obtain data down to the specific item level. Item analysis reports allow teachers to see the standard assessed by each item and the percentage of students that chose each answer option for each assessment item. Reports can also be created to view students' progress over a defined period of time.

Districts can decide whether to run the program on a locally based Web-accessible network or Renaissance Learning can host the network and make it available to the district. Most user districts are currently hosting the program on their own networks.

In terms of the product's content and item reliability, Renaissance Learning employs experienced items writers, who have backgrounds in education. Many have been teachers. Also, many have written content for textbook publishers and similar educationally oriented companies. In addition, the company had test-preparation products that contain content, and most of that content has been repurposed for use in the StandardsMaster product, a company official said.

The company is currently engaged in research evaluating the reliability and validity of StandardsMaster assessments and collecting data from users regarding their performance on high-stakes tests, and is planning to make such results available in the future.

StandardsMaster is sold on a per-student per-year subscription basis. There is one fee for the software, and a fee for each additional content area selected by a district. Total cost is dependent on a number of factors, such as size of the district. Renaissance Learning also offers data conversion services to get all current student information into the system.

Assess2Learn is an assessment product that delivers comprehensive exams to schools via the Internet. Geared to grades 1-8, it is the online assessment product of Illinois-based Riverside Publishing, a division of Houghton Mifflin Company and a major test publisher that offers various educational testing products and services. Assess2Learn assesses students' skills in reading, language arts, and math relative to state and national standards and provides immediate, real-time test results.

As a Web-based diagnostic tool, Assess2Learn is both an assessment content product aligned to state and local standards and the engine for delivering the assessment. The company has developed a test bank, and has created three test "forms" for each subject area that can be accessed by teachers. Using the Assess2Learn tool, teachers and educators can determine where students are strong and weak on skills defined by state and local standards; tailor in-depth instruction specifically for each student; and identify at-risk students and populations on the class, grade, and school level. With Assess2Learn, users such as teachers, administrators, or curriculum specialists also have the ability to create tests. The test author would devise the tests, the sections, the questions, the answer choices, and the grading standards.

Riverside works with customers in exactly how they want to use Assess2Learn and its content. The company will sell schools its own content if they already have something they're using; it will sell them usage of the Assess2Learn engine if they have their own content to place within the engine; or Riverside will sell them both the content and the usage of the engine. The third option does not limit schools from loading their own content to supplement the company's.

Assess2Learn is sold primarily to schools and districts. It is currently used in 86 districts, and the Assess2Learn tests are available to more than 800,000 students. The company has also been awarded the contract by the state of Georgia to develop a statewide assessment tool for grades 1-8.

The assessment product is done on an ASP model in which a school's student information data is first imported into the system. Riverside does all of the data hosting, although if districts want to import test-results data on to their servers, they can. Students log on, take the appropriate tests, and click a button for the test to be graded as soon as they are finished; the results are immediately scored. The data is saved on the ASP server. Test results are scored and displayed according to the requirements specified by schools during the assignment process. Results are available immediately to students, teachers, and school and district administrators.

Assess2Learn provides various comprehensive test result reports to teachers and administrators. These include the ability to drill down data and see skill proficiency reports by student, by class, by building, and by school system. Results can also be used to break out each state standard and measure how students did against the standard. Item analysis reports permit educators to see how students performed on each item.

Currently, the assessment content is aligned to standards in 14 states, with another state due to be added in the next year. There is also a national edition available to schools. The company's goal is to add content aligned to an additional 12 states each year, until the entire country is covered, a company official said.

Riverside is currently adding in the features that would allow data to be viewed based on No Child Left Behind requirements. Data can currently be imported through these filters, but the system does not yet generate reports along such categories. This functionality is expected to be ready by August, a company official noted.

Riverside Publishing develops and publishes criterion-referenced, norm-referenced, and ability tests, as well as alternative assessments that enable school personnel to measure the educational progress of students. This test-development experience is brought to bear in the creation and maintenance of the Assess2Learn item pool. There are 25,000 test items available to be used in the Assess2Learn assessments.

The company has a staff of test development specialists who oversee item writers and who have experience and training in many fields of education, numerous content areas, psychometrics, and management. The model for the electronic testing development process includes seven steps: review the state's guiding documents and develop content domain specifications; develop test blueprint specs; develop item specifications; select and train item writers; develop test items; conduct several stages of internal item review for content, bias, and sensitivity; and produce online test forms.

The company's test development specialists are experienced item writers. They also use a team of consultant item writers who have composed items for other Riverside products and who are teachers or former teachers with specialized knowledge concerning their content areas of expertise.

There are several stages of internal item review. In the review process, the items are reviewed and verified for their alignment to state standards; accuracy; appropriateness for difficulty pertaining to the purpose of the test; relevance; and fairness. They are reviewed to be sure that they are free of bias and sensitivity issues. A content specialist and a senior reviewer review all test items for potential bias.

Riverside focuses on content validity through the detailed process outlined above (under "Item Pool") in the creation of its online assessments. Once the test is created and put online, a thorough QA process is utilized to ensure content validity; that each item measures the standard it is purported to measure; and that answer keys are correct, etc.

In addition, the company is gathering both quantitative and qualitative data to ensure items perform in an acceptable manner, and the company makes ongoing changes as appropriate. Quantitative data is gathered as the tests are being administered to students, while qualitative data is obtained based on feedback from educators in the various states where the company publishes content. The company also examines other outcome measures. The diagnostic nature of the tests provides detailed information down to the granular level of the standards, a company official said.

Assess2Learn is sold on a per student per year basis. This cost is divided into two components. There is one segment of the cost that is for the technology - for the Assess2Learn engine itself. And, there is a one-time fee for the Assess2Learn content. The technology subscription cost typically runs $6 per student per year. The content cost is a one-time charge of $3 per student per content form.

The Performance Series is Scantron's Internet-delivered standards-based adaptive measurement - a computer adaptive assessment created to measure the different academic strands of individual state standards. It aims to give immediate diagnostic information to each teacher, including learning objectives a student has not completed, groupings of students by ability, and the academic gains made by individual and groups of students. Scanton's Performance Series assessments are for grades 2-12. The company has been offering Reading, Math, and Learning Styles assessments for the past two years. Science and Language Arts modules have also been developed and will be tested beginning this spring. Since the product's launch about two years ago, some 1.5 million students have been tested on these assessments.

The EdVision-developed computer adaptive assessment targets the instructional level of each student by altering question difficulty based on previous answers. It is done on an ASP model, which is delivered via the Web, with all data hosted by Scantron. Schools can test students as long as they have a computer with Internet access for the student. Educators can then view the results of an assessment on the Web immediately.

This diagnostic assessment places a student at his or her appropriate instructional level immediately. It is geared toward driving instruction and helping teachers, through its findings, to create a personalized learning plan for each student, based on his or her performance on the adaptive assessment. Such adaptive testing is different than levels testing in that a whole class will not receive the same assessment, and it is based on the recognition that students in one class actually are situated in many different grade levels, skills-wise.

The Scantron standards-based adaptive measurement tracks students using a consistent scale as they move from one school to the next, and measures academic growth over one year or across multiple years. Teachers get very detailed reports of a student's results that indicate what would be the next step to address specific skills weaknesses. Administrators can access scores for the whole district, as well as for each school. The data can be aggregated and disaggregated, so that educators can group data and obtain reports in a variety of filters, such as gender, students with disabilities, etc., which help districts fulfill objectives of NCLB.

The Scanton assessments are research-based. The items are internally created and go through peer review and bias testing. When the company's content team creates an item, the item goes out for external review by a team of consultants, made up of educators (ex-teachers, teachers, and others with educational background.) Once the item is checked, if it needs to be revised, the item is sent out for review again and to go through an approval process. Items are tested on a pilot site, and the company's statisticians review the data with regard to the performance of the item.

The concept of reliability for an assessment instrument can be thought of as the degree to which an assessment instrument provides consistent measures. Scantron utilizes two methods of illustrating this concept statistically. At the beginning of the project, test items were examined in a non-adaptive manner. To do this, items were grouped into non-adaptive tests, where, for example all 5th grade math questions only are grouped together and administered to groups of 5th graders only. Under this non-adaptive method, there are three considerations: an item's difficulty, its ability to discriminate between students with high ability and low ability, and how all of the items on the non-adaptive test work together.

This third concept is sometimes referred to as internal consistency reliability, and is given as a coefficient value that is most commonly referred to as a KR-20 or Cronbach Alpha value. This statistic indicates how all items on a test relate to each other and to the test as a whole. The theoretical maximum is +1, and a higher score has an interpretation of a higher degree of reliability for the test.

The second method used accommodates the Computer Adaptive format of the Performance Series. A statistic called the Standard Error of Measurement is calculated in this case, as there is no "set" of test items seen by a large group of people for which a KR-20 or Cronbach's Alpha value can be calculated. This Standard Error of Measurement value is calculated for an individual and can be thought of as a confidence interval within which an individual's measure will fall with repeated assessments. A small Standard Error of Measurement implies that repeated assessments for an individual would provide measures of ability for the individual that are consistent.

The cost of the Performance series assessments is on a per-student per-year basis, and can differ somewhat due to the size of the district and other options. Typically, it runs about $15 per student, which includes unlimited testing and all reports. Scantron also sends out service technicians to work with schools' IT departments, to validate that the school's technical infrastructure - firewall, content filters, browser levels - are suitable and helping to complete the data importing.

TestU, a company that concentrates most of its efforts on test prep programs for grades 9-12, recently launched a new product, Middle School Math Pathways, which is a standards-based math assessment program. The Math Pathways Internet-based assessments are aligned with state content standards. The program is a "pure assessment product": designed to help school districts assess their students' progress on state content standards before they reach high school, says a company official. It provides real-time data.

For each year in grades 5-8, students are administered five required tests to assess performance on core skills. Three types of assessments are in the tests: readiness assessment, which assesses "back to school" preparedness at the beginning of the year; formative assessment, administered at the end of the first quarter, which assesses concepts covered in that time period and provides a snapshot of student progress; and summative assessment, which covers topics from the entire year or semester and reveals student mastery of the curriculum.

TestU also offers a concentration of online tools in grades 9-12 geared toward practice for states' high school exit and high school proficiency exams. The programs combine diagnostic assessments, preparatory courses, and practice tests. High school exit exams covered range from the New York Regents Math and English and Texas TEKS to Florida's FCAT and the California High School Exit Exam. The company's Skill Navigator product is designed to aid students in other states meet requirements.

For their products, the company both generates its own test items, working with consultants who have written content for particular states, and it also licenses content from sources such as Barron's. There are more than 15,000 test items in the TestU pool.

Training sessions are incorporated in these TestU products. The packages sold to school contain one or two days of training, depending on what is purchased. In addition, the company has a team of coaches - who are current or former teachers - working with schools on an ongoing basis.

TestU formerly sold its tools on a per-student basis, but now bases prices per school site or per classroom. There is a range of prices, but typically, a package can be purchased for a class with 30 students for $4,500, including two days of training. The cost for a single high school site is typically $7,500, including two days of training.