[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Eur. Patent auf computergestuetzte Lernstoff-Pruefungen



EP0664041
System and method for computer based testing
EDUCATIONAL TESTING SERVICE (US)

phm: Der Hauptanspruch deckt alle notwendigen Merkmale der
computergestuetzten Durchfuehrung einer Pruefung in Schule oder Uni ab.  
Weitere Varianten stehen in den Unteranspruechen.  Ganz nach dem Muster
der EPA-Patente "Erzeugung eines Einkaufszettels aus Kochrezepten" und
"digitales Sprachlernsystem".

Ich frage mich, ob unsere Kultusministerien schon bei ETS Inc
Lizenzgebuehren abfuehren oder wenigstens zum Dank fuer eine kostenlose
Lizenz Werbeplakate fuer ETS in deutschen Schulen anbringen.

   Claims
   
   We claim:
   1. A computer based testing system comprising: a test development
   system for creating a computerized test having a number of questions
   to be answered by an examinee, a plurality of related test screens
   containing messages and directions providing information for the
   examinee, and a computerized test script defining rules for
   determining a sequence of questions and related test screens to be
   presented; a workstation operable to present the questions to at least
   one said examinee and operable to accept examinee responses to the
   questions so presented; and a test delivery system operatively coupled
   to both said test development system and said workstation for
   delivering said computerized test by presenting the questions and
   related test screens according to the rules defined by the test script
   to each said examinee on said workstation.
   2. The system of claim 1, further comprising: a test administration
   system operatively coupled to said test development system, said test
   delivery system and said workstation for initiating and terminating
   the delivery of said computerized test by said test delivery system.
   3. The system of claim 2, wherein said test administration system
   further comprises security means for securing said computerized test
   from access by unauthorized persons.
   4. The system of claim 2, wherein a plurality of events indicative of
   errors of said computer based testing system and related to security
   of said computerized test and said computer based testing system are
   monitored by said test administration system, and a plurality of
   predetermined events occurring during the delivery of said
   computerized test are monitored by said test delivery system, each
   event and related information being stored within a log record, a
   combination of said log records storing events related to errors
   defining an error log file, a combination of said log records storing
   events related to the security of said computerized tests and computer
   based testing system defining a security log file, and a combination
   of said log records storing events occurring during the delivery of
   said computerized test defining an examinee performance file.
   5. The system of claim 4, wherein said workstations are located at at
   least one test center, the system further comprising: a data
   distribution system having an interface with said test administration
   system for processing said error log files, said security log files
   and said examinee performance files and for generating from these
   files reports indicative of events occurring at each workstation and
   at each of said test centers.
   6. The system of claim 1, wherein said test development system further
   comprises: a test document creation means for defining the questions
   to be presented to the examinee based on a test specification and for
   determining a manner of presentation of the questions on said
   workstation during the delivery of said computerized test; an item
   preparation system interfaced with said test document creation means
   for computerizing the questions for delivery by said test delivery
   system; a test preparation system interfaced with said test document
   creation means and said item preparation system for preparing the
   computerized test script and the related test screens; and a test
   packaging utility interfaced with said item and test preparation
   systems for combining the questions, the test script and the related
   test screens to form the computerized test.
   7. The system of claim 6, wherein the test specification specifies a
   number of test sections of the computerized test in which the
   questions are assigned, each test script defining the rules for
   determining the sequence of questions and related test screens to be
   presented in one of said test sections and wherein said test
   preparation system is operative to create, and the test delivery
   system is operative to deliver, at least one of: tutorials for
   providing instructions to the examinee for interacting with said
   workstation during the delivery of said computerized test; directions
   for explaining how to take said computerized test; a help facility
   selectable by the examinee for accessing additional instructions
   describing how to interact with said workstation and for accessing
   said directions for explaining how to take the computerized test; a
   review facility selectable by the examinee during the delivery of said
   computerized test for providing a list of substantially all of the
   questions of said computerized test and for allowing the examinee to
   select a specific question from the list to be presented; and said
   test preparation system creating a session script defining a sequence
   in which the tutorials, the directions, and the test scripts are to be
   invoked by the test delivery system and wherein the test delivery
   system is operable to interrupt the sequences defined by the session
   script and the sequences determined based on the rules defined by the
   test scripts to invoke the help and review facilities upon selection
   by the examinee.
   8. The system of claim 1, wherein a plurality of predetermined events
   are monitored by said test delivery system, each event and related
   information being stored within a log record, a combination of log
   records created during the delivery of said computerized test defining
   a performance file.
   9. The system of claim 8, wherein said events include said examinee
   responses, the system further comprising: a data distribution system
   having an interface with said test delivery system for formatting said
   performance files so that said examinee responses can be evaluated.
   10. The system of claim 1, wherein said computerized test is a
   standardized test.
   11. The system of claim 1, wherein said workstations are located at at
   least one test center, the system further comprising: a data
   distribution system located at a central processing site remote from
   said test centers, said data distribution system having a data
   connection with each said test center for receiving information
   related to activities at the test centers and information indicative
   of said examinee responses and for processing said information to
   generate at least one report presenting said information in human
   readable form and for formatting at least some of said information for
   subsequent scoring of examinee responses.
   12. The system of claim 1, wherein the rules for determining the
   sequence of questions to be presented are based on a measurement model
   for at least one of a linear test, an adaptive test, a randomized
   parallel test, and a mastery test.
   13. A method of computer based testing, comprising the steps of:
   producing a computerized test having a plurality of questions to be
   answered by an examinee, a plurality of related test screens
   containing messages and directions comprising information for the
   examinee, and a computerized test script defining rules for
   determining a sequence of the questions and related test screens to be
   presented; delivering said computerized test to the examinee by
   presenting said plurality of questions and related test screens to the
   examinee at a computer workstation according to the rules defined by
   the test script; and recording responses provided by the examinee to
   the questions presented.
   14. The method of claim 13, further comprising the step
   of:administering at least one said computerized test to at least one
   examinee.
   15. The method of claim 13, further comprising the step of: securing
   said computerized test and responses from access by an unauthorized
   person.
   16. The method of claim 13, wherein said step of delivering said
   computerized test further comprises the step of: providing user
   controls selectable by the examinee for controlling said delivery of
   said computerized test.
   17. The method of claim 13, wherein said workstation comprises a
   memory, the method further comprising the steps of: registering at
   least some examinees to take a specified computerized test prior to
   the delivery of the computerized test, each examinee so registered
   being authorized to take said specified computerized test; storing at
   least one specified computerized test in said memory prior to delivery
   of said specified computerized test.
   18. The method of claim 17, further comprising the step of:
   safeguarding each computerized test stored in said memory from access
   by unauthorized persons.
   19. The method of claim 17, further comprising the step of: confirming
   the authorization of each examinee seeking to take said specified
   computerized test prior to delivering said specified computerized
   test.
   20. The method of claim 17, further comprising the steps of: selecting
   one specified computerized test stored in memory to be delivered, the
   test selected being said specified computerized test the examinee is
   authorized to take; and initiating the delivery of said computerized
   test so selected.
   21. The method of claim 13, wherein said computerized test is
   standardized and wherein said computerized test is based on a test
   specification being defined independent of said test development
   system used to produce the computerized test.
   22. The method of claim 21, wherein the test specification specifies a
   number of test sections of the computerized test in which the
   questions are assigned, each test script defining rules for
   determining the sequence of questions and related test screens to be
   presented in one of said test sections, wherein said step of
   delivering said computerized test further comprises at least one of
   the following steps: displaying tutorials to the examinee for
   instructing the examinee how to interact with said workstation;
   displaying directions related to the questions to explain how the
   examinee is intended to respond to each question presented; providing
   a help facility selectable by the examinee to access instructions
   describing how to interact with said workstation and for accessing
   directions describing how to respond to questions presented; and
   providing a review facility selectable by the examinee to review
   questions presented by providing a list of substantially all questions
   included in said computerized test and for allowing the examinee to
   return and to advance to a question listed; and said method further
   comprising the steps of: creating a session script defining a sequence
   in which the tutorials, the directions, and the test scripts are to be
   invoked; and interrupting the sequence defined by the session script
   and the sequence determined based on the rules defined by the test
   scripts to invoke the help and review facilities upon selection by the
   examinee.
   23. The method of claim 13, wherein each said workstation is located
   at a test center.
   24. The method of claim 23, further comprising the steps of:
   transferring information indicative of activities occurring at each
   said test center from said test centers to a central processing site
   being remote from said test centers; and processing said transferred
   information so that said activities at said test centers are monitored
   at said central processing site.
   25. The method of claim 23, further comprising the steps of:
   transferring said recorded responses from each workstation to a
   central processing site being remote from said test centers; and
   formatting said recorded responses for subsequent scoring.
   26. The method of claim 13, wherein the rules for determining the
   sequence of questions to be presented are based on a measurement model
   for at least one of a linear test, an adaptive test, a randomized
   parallel test, and a mastery test.
   27. A method of producing a computerized test, comprising the steps
   of: preparing a test document of items, each item having at least a
   test question, related presentation information indicative of at least
   the text to be presented to an examinee and a response type associated
   with said item; preparing an electronic form of each item according to
   said presentation information and said response type and defining said
   electronic form so prepared as a computerized item; preparing at least
   one session script having information indicative of predetermined test
   options and information indicative of a sequence in which said
   computerized items are to be presented; and packaging said
   computerized items and said session scripts together to form said
   computerized test.
   28. The method of claim 27, wherein computerizing each item comprises
   the steps of: editing the items according to the presentation
   information using a word processor; and inserting custom codes into
   the items according to both the presentation information and the
   response type.
   29. The method of claim 27, wherein the items further comprise
   stimulus material to be referenced by the items, the method further
   comprising the step of: demarcating blocks of the stimulus material by
   inserting custom codes into the stimulus material; said insertion of
   the custom codes causing the demarcated blocks to be one of i)
   highlighted upon presentation to the examinees, and ii) selectable by
   the examinees.
   30. The method of claim 27, wherein each computerized test has at
   least one test section, each test section having at least one item,
   further comprising the steps of: preparing for each said section, at
   least one delivery unit defining options to be effective for that said
   section and for specifying an organization of said computerized items
   to be presented within that said section.
   31. The method of claim 30, further comprising the steps of: preparing
   at least one of the following units: (1) a general information screen
   (GIS) unit having information related to at least one of said test,
   said section, and said item, and (2) a tutorial unit having
   information to familiarize examinees with said computerized test;
   combining GIS units, tutorial units and delivery units to form a
   testing unit; and generating a test script defining a sequence in
   which said GIS units, tutorial units and delivery units are to be
   presented, at least one test script being included within each of said
   session scripts.
   32. The method of claim 31, further comprising the steps of: preparing
   at least one of the following units: (1) an examinee data collection
   unit having at least one inquiry to be presented to examinees to
   obtain information about said examinees, and (2) a scoring and
   reporting unit having information related to scoring and reporting of
   at least one computerized test; and defining a sequence in which said
   examinee data collection units, scoring and reporting units, testing
   units, GIS units, and tutorial units are to be presented, said session
   script being based on said sequence.
   33. The method of claim 27, further comprising the step of:generating
   computerized graphics to be presented with the computerized items.
   34. The method of claim 27, wherein each item has a key indicative of
   a correct response to the test question associated therewith, the
   method further comprising the step of: computerizing the key; and
   packaging the computerized key with the computerized test.
   35. The method of claim 27, further comprising the step of: presenting
   each computerized item for review; and modifying the computerized item
   based on said presentation.
   36. A computer based testing system comprising: a test development
   system for creating a computerized test for standardized testing, said
   computerized test having a number of questions to be answered by an
   examinee and being based on a test specification which specifies at
   least one of the following information: the number of questions to be
   presented in the computerized test, the number of test sections in
   which the questions are grouped, the number of questions in each
   section, an amount of time for taking the test, an amount of time
   allotted for responding to each of the questions, and criteria for
   selecting questions to be presented, said test specification being
   defined independent of said test development system; a workstation
   operable to present the questions to at least one said examinee and
   operable to accept examinee responses to the questions so presented;
   and a test delivery system operatively coupled to both said test
   development system and said workstation for delivering said
   computerized test to each said examinee on said workstation in a
   manner independent of the information specified by the test
   specification.
   37. The system of claim 36, further comprising: a test administration
   system operatively coupled to said test development system, said test
   delivery system and said workstation for initiating and terminating
   the delivery of said computerized test by said test delivery system.
   38. The system of claim 37, wherein said test administration system
   further comprises security means for securing said computerized test
   from access by unauthorized persons.
   39. The system of claim 37, wherein a plurality of events indicative
   of errors of said computer based testing system and related to
   security of said computerized test and said computer based testing
   system are monitored by said test administration system, and a
   plurality of predetermined events occurring during the delivery of
   said computerized test are monitored by said test delivery system,
   each event and related information being stored within a log record, a
   combination of said log records storing events related to errors
   defining an error log file, a combination of said log record storing
   events related to the security of said computerized tests and computer
   based testing system defining a security log file, and a combination
   of said log records storing events occurring during the delivery of
   said computerized test defining an examinee performance file.
   40. The system of claim 39, wherein said workstations are located at
   at least one test center, the system further comprising: a data
   distribution system having an interface with said test administration
   system for processing said error log files, said security log files
   and said examinee performance files and for generating from these
   files reports indicative of events occurring at each workstation and
   at each of said test center.
   41. The system of claim 36, wherein said test development system
   further comprises: a test document creation means for defining the
   questions to be presented to the examinee and for determining a manner
   of presentation of the questions on said workstation during the
   delivery of said computerized test; an item preparation system
   interfaced with said test document creation means for computerizing
   the questions for delivery by said test delivery system; a test
   preparation system interfaced with said test document creation means
   and said item preparation system for preparing a computerized test
   script defining a sequence of the questions to be presented and for
   preparing related test screens containing of messages and directions
   to be presented during the delivery of the computerized test; and a
   test packaging utility interfaced with said item and test preparation
   systems for combining the test script and related test screens to form
   the computerized test.
   42. The system of claim 41, wherein each test script defines rules for
   determining the sequence of questions and related test screens to be
   presented in one of said test sections and wherein said test
   preparation system is operative to create and said test delivery
   system is operative to deliver at least one of: tutorials for
   providing instructions to the examinee for interacting with said
   workstation during the delivery of said computerized test; directions
   for explaining how to take said computerized test; a help facility
   selectable by each examinee for accessing additional instructions
   describing how to interact with said workstation and for accessing
   said directions for explaining how to take the computerized test; a
   review facility selectable by each examinee during the delivery of
   said computerized test for providing a list of substantially all of
   the questions of said computerized test and for allowing the examinee
   to select a specific question from the list to be presented; and said
   test preparation system creating a session script defining a sequence
   in which the tutorials, the directions, and the test scripts are to be
   invoked by the test delivery system and wherein the test delivery
   system is operable to interrupt the sequence defined by the session
   script and the sequences determined based on the rules defined by the
   test scripts to invoke the help and review facilities upon selection
   by the examinee.
   43. The system of claim 42, wherein the rules for determining the
   sequence of questions to be presented are based on a measurement model
   for at least one of a linear test, an adaptive test, a randomized
   parallel test, and a mastery test.
   44. The system of claim 36, wherein a plurality of predetermined
   events are monitored by said test delivery system, each event and
   related information being stored within a log record, a combination of
   log records created during the delivery of said computerized test
   defining a performance file.
   45. The system of claim 44, wherein said events include said examinee
   responses, the system further comprising: a data distribution system
   having an interface with said test delivery system for formatting said
   performance files so that said examinee responses can be evaluated.
   46. The system of claim 36, wherein said workstations are located at
   at least one test center, the system further comprising: a data
   distribution system located at a central processing site remote from
   said test centers, said data distribution system having a data
   connection with each said test center for receiving information
   related to activities at the test centers and information indicative
   of said examinee responses and for processing said information to
   generate at least one report presenting said information in human
   readable form and for formatting at least some of said information for
   subsequent scoring of examinee response.
   47. A method of computer based testing, comprising the steps
   of:producing a computerized test for standardized testing using a test
   development system, said computerized test having a plurality of
   questions to be answered by an examinee and being based on a test
   specification which specifies at least of the following information:
   the number of questions to be presented in the computerized test, the
   number of test sections in which the questions are grouped, the number
   of questions in each section, an amount of time for taking the test,
   an amount of time allotted for responding to each of the questions,
   and criteria for selecting questions to be presented, said test
   specification being defined independent of said test development
   system; delivering said computerized test to the examinee by
   presenting said plurality of questions to the examinee at a computer
   workstation in a manner independent of the information specified by
   the test specification; and recording responses provided by the
   examinee to the questions presented.
   48. The method of claim 47, further comprising the step of:
   administering at least one said computerized test to at least one
   examinee.
   49. The method of claim 47, further comprising the step of: securing
   said computerized test and responses from access by an unauthorized
   person.
   50. The method of claim 47, wherein said computerized test comprises a
   plurality of related test screens containing messages and directions
   providing information for the examinee and a computerized test script
   defining rules for determining a sequence of the questions and related
   test screens to be presented and wherein said step of delivering said
   computerized test further comprises at least one of the following
   steps: displaying tutorials to the examinee for instructing the
   examinee how to interact with said workstation; displaying directions
   related to the questions to explain how the examinee is intended to
   respond to each question presented; providing a help facility
   selectable by the examinee to access instructions describing how to
   interact with said workstation and for accessing directions describing
   how to respond to questions presented; and providing a review facility
   selectable by the examinee to review questions presented by providing
   a list of substantially all questions included in said computerized
   test and for allowing the examinee to return and to advance to a
   question listed; and said method further comprising the steps of:
   creating a session script defining a sequence in which the tutorials,
   the directions, and the test scripts are to be invoked; and
   interrupting the sequence defined by the session script and the
   sequences determined based on the rules defined by the test scripts to
   invoke the help and review facilities upon selection by the examinee.
   51. The method of claim 50, wherein the rules for determining the
   sequence of questions to be presented are based on a measurement model
   for at least one of a linear test, an adaptive test, a randomized
   parallel test, and a mastery test.
   52. The method of claim 47, wherein said step of delivering said
   computerized test further comprises the step of: providing user
   controls selectable by the examinee for controlling said delivery of
   said computerized test.
   53. The method of claim 47, wherein said workstation comprises a
   memory, the method further comprising the steps of: registering at
   least some examinees to take a specified computerized test prior to
   the delivery of the computerized test, each examinee so registered
   being authorized to take said specified computerized test; storing at
   least one specified computerized test in said memory prior to delivery
   of said specified computerized test.
   54. The method of claim 53, further comprising the step of:
   safeguarding each computerized test stored in said memory from access
   by unauthorized persons.
   55. The method of claim 53, further comprising the step of: confirming
   the authorization of each examinee seeking to take said specified
   computerized test prior to delivering said specified computerized
   test.
   56. The method of claim 53, further comprising the steps of: selecting
   one specified computerized test stored in memory to be delivered, the
   test selected being said specified computerized test the examinee is
   authorized to take; and initiating the delivery of said computerized
   test so selected.
   57. The method of claim 47, wherein each said workstation is located
   at a test center.
   58. The method of claim 57, further comprising the steps of:
   transferring information indicative of activities occurring at each
   said test center from said test centers to a central processing site
   being remote from said test centers; and processing said transferred
   information so that said activities at said test centers are monitored
   at said central processing site.
   59. The method of claim 57, further comprising the steps of:
   transferring said recorded responses from each workstation to a
   central processing site being remote from said test centers: and
   formatting said recorded responses for subsequent scoring.
     _________________________________________________________________
   
   Data supplied from the esp@cenet database - l2