Telehealth and the D-KEFS

The telehealth information in this document is intended to support psychologists in making informed, well-reasoned decisions around remote assessment. This information is not intended to be comprehensive regarding all considerations for assessment via telehealth. It should not be interpreted as a requirement or recommendation to conduct assessment via telehealth.

Psychologists should remain mindful to:

Follow professional best practice recommendations and respective ethical codes
Follow telehealth regulations and legal requirements from federal, state, and local authorities; licensing boards; professional liability insurance providers; and payors
Develop competence with assessment via telehealth through activities such as practicing, studying, consulting with other professionals, and engaging in professional development

Psychologists should use their clinical judgment to determine if assessment via telehealth is appropriate for a particular examinee, referral question, and situation. There are circumstances where assessment via telehealth is not feasible and/or is contraindicated. Documentation of all considerations, procedures, and conclusions remains a professional responsibility.

Several professional organizations and experts have provided guidance on telehealth assessment (American Psychological Association Services [APA Services], 2020; Grosch et al., 2011; Inter Organizational Practice Committee, 2020; Stolwyk et al., 2020) to assist psychologists in decision making and ethical and legal practice issues.

The Delis–Kaplan Executive Function System (D–KEFS™; Delis, Kaplan, & Krammer, 2001) is a set of nine, higher level cognitive tests that require examiner instructions and feedback throughout administration. The D–KEFS can be administered in a telehealth context by using digital tools from Q-global®, Pearson’s secure online testing and scoring platform. Q-global provides digital assets (e.g., the D–KEFS stimulus book) that can be shown to the examinee via the screen-sharing features of its teleconference platform. Details regarding Q-global and how it is used are provided on the Q-global product page.

Several of the D–KEFS tests lend themselves well to telehealth evaluations because the only visual stimuli shown are written sentences and only require the examinee to make verbal responses. These measures include the D–KEFS Verbal Fluency Test, Word Context Test, and Proverb Test. In addition, the Color–Word Interference Test and Twenty Questions Test may be administered remotely if the stimuli can be displayed accurately; however, this requires verification of orientation, color, and font size. Other D–KEFS tests are not easily adapted to telehealth evaluations because they require the examinee to draw on record forms or use manipulatives. These measures include the D–KEFS Trail Making Test, Design Fluency Test, Sorting Test, and Tower Test. For these tests, a professional or nonprofessional facilitator may be needed to optimize assessment accuracy, especially for examinees with lower cognitive abilities.

Although some options are available for administering the D–KEFS via telehealth, it is important to consider that the normative data were collected via face-to-face assessment. Telehealth is a deviation from the standardized administration, and the methods and approaches used in telehealth administration should be supported by research and practice guidelines when appropriate.

Providers engaging in telehealth assessment may train facilitators to work with them on a regular basis in order to provide greater coverage to underserved populations (e.g., only two providers within a 500-mile radius, shortage of school psychologists within a school district). If such a facilitator is well trained and in a professional role (i.e., a professional facilitator), they can present the response booklet as well as adjust audiovisual equipment. While this approach yields the same D–KEFS scores that are available in face-to-face assessment mode, clinical judgment is required to determine the validity of this administration method given the lack of direct evidence for the equivalence of D–KEFS scores across modalities. If a professional facilitator is not used, it may increase measurement error, particularly for some tests, and therefore may impact the validity of the derived scores.

A key variable in determining whether or not a trained facilitator is needed for telehealth assessment is the level of cognitive functioning of the examinee. For younger children, older adults, and individuals with significant neurological or psychiatric disorders, a trained facilitator will likely be needed, especially for tests that require drawing on record forms or use of manipulatives. In contrast, for individuals with higher levels of overall cognitive functioning, a trained facilitator may not be needed. Again, clinical judgment is required to determine the need for using a trained facilitator; the decision of whether or not to use a facilitator and its impact on the validity of the assessment should be explicitly addressed in the practitioner’s report.

Another variable in determining whether or not it is safe or feasible to use a trained facilitator is the need for social distancing (e.g., during a recent outbreak of COVID-19). If testing must occur under these conditions, it is possible that the examinee may participate without the help of an onsite facilitator. If the examiner determines that a facilitator is not required, the examinee can assist with technological and administrative tasks during testing and should be oriented to these responsibilities prior to, and again at the beginning, of the session. An initial virtual meeting should occur in advance of the testing session to address numerous issues specific to testing via telehealth. This initial virtual meeting is described in the administrative and technological tasks portion of the Examiner Considerations section and referred to in various sections of this document. The examiner should consider best practice guidelines, the referral question, and the examinee’s level of cognitive functioning and overall clinical condition, as well as telehealth equivalence study conditions, to determine if this is possible and appropriate. Again, independent examinee participation may not be possible or appropriate, for example, for examinees with lower cognitive ability or with lower levels of technological literacy and experience.

If the examiner determines that the examinee cannot participate independently, and testing must occur under social distancing constraints, another option is to enlist the assistance of someone in the examinee’s home (e.g., a parent, guardian, or caretaker). If the onsite facilitator is not in a professional role (i.e., nonprofessional facilitator), they can assist with technological and administrative tasks during testing and should be oriented to these responsibilities in the initial virtual meeting and again at the beginning of the session.

Professional and nonprofessional facilitators should be in the testing room only for those tests that require their ongoing assistance (e.g., the Tower Test). The examiner should plan to minimize (as much as possible) the need for the facilitator to remain in the room. In rare cases when the facilitator must remain in the room, they should do so passively and unobtrusively, and merely to monitor and address the examinee’s practical needs, as well as any technological or administrative issues as necessary. The facilitator’s role should be defined clearly by the examiner before the testing session. The facilitator should only perform those functions the examiner approves and deems necessary. The extent to which the facilitator assisted with the administration of the tests should also be discussed in the practitioner’s report. In any case, if a facilitator is necessary, it is preferred that the facilitator remain accessible.

The response booklets contain the response forms for the Trail Making and Design Fluency tests. Response booklets are required for the administration of these tests. Because of the timing and immediate corrective feedback requirements of these tests, these subtests should be administered with a trained facilitator for most examinees, except for those individuals with higher levels of cognitive skills. Again, the practitioner’s clinical judgment is needed to make this determination, and the assessment procedures used should be discussed in the report.

Delis-Kaplan Executive Function System (D-KEFS)

See pricing options

Conducting Telehealth Assessment

Conducting a valid assessment in a telehealth service delivery model requires an understanding of the interplay of a number of complex issues. In addition to the general information on Pearson’s telehealth page, examiners should address five factors (adapted from Eichstadt et al., 2013) when planning to administer and score assessments via telehealth.

1. Telehealth Environment & Equipment

Computers and connectivity

Two computers with audio and video capability and stable Internet connectivity—one for the examiner and one for the examinee—are required. A web camera, microphone, and speakers or headphones are required for both the examiner and the examinee. A second computer screen or split-screen format on a large computer monitor for the examiner is helpful to allow a view of the digital administration stimulus book and administration manual, but the examiner can also use the paper format manual/stimulus book or the Q-interactive® platform for Trail Making, Verbal Fluency, Design Fluency, and Color–Word Interference. The second computer monitor or large screen also tends to make sharing test content more straightforward for the examiner.

Image/screen size

When items with visual stimuli are presented, the digital image of the visual stimuli on the examinee’s screen should be similar in size to the paper stimulus book (8.5 x 11). Twenty Questions requires the larger screen size because of the size and placement of the images. Some teleconferencing platforms shrink the size of images, so the facilitator should verify the image size in the initial virtual meeting. It is recommended that computer screens used for teleconference assessment be at least 15” measured diagonally. Smaller screens, such as those of the iPad®, iPad mini®, small tablet PCs, and smartphones, are not allowed for examinee-facing content because these have not been examined empirically and may affect stimulus presentation, examinee response, and validity of the test results. Similarly, presenting stimuli on extremely large screens has not been examined, so the same precaution applies. At the beginning of the testing session, the examiner may ask for a peripheral camera or device (as described later in this section) to be aimed at the examinee’s screen to ensure that the examinee’s screen is displaying images in the correct aspect ratio and not stretching or obscuring the stimuli image.

It is important to note that, in traditional in-person assessments, the stimuli for the Color–Word Interference and Twenty Questions tests are presented flat on the table. There is no research investigating differences between presentation formats on performance at the time of this report. While there is no evidence suggesting that the angle of presentation of these stimuli should significantly affect the validity of the findings, this change in standardized administration should be noted in the practitioner’s report along with any observations on whether or not this change may have affected the validity of the findings.

Teleconference platform

A teleconference platform is required. Screensharing capability is required if anything other than items with verbal stimuli and responses are administered.

Video

High-quality video (HD preferred) is required during the administration. Make sure the full faces of the examiner and the examinee are seen using each respective web camera. The teleconference platform should allow all relevant visual stimuli to be fully visible to the examinee when providing instruction or completing items; the view of the examiner should not impede the examinee’s view of visual test stimuli.

Screensharing digital components

Digital components are shared within the teleconferencing software as specified in Table 1. There are two ways to view digital components in the Q-global Resource Library: through the pdf viewer in the browser window or full screen in presentation mode. Always use full-screen (i.e., presentation) mode for digital components viewed by the examinee. This provides the cleanest presentation of test content without onscreen distractions (e.g., extra toolbars). Refer to Using Your Digital Assets on Q-global in the Q-global Resource Library for complete directions on how to enter presentation mode.

Test item security in the audiovisual environment

The examiner is responsible for ensuring test item security is maintained, as outlined in the Terms and Conditions for test use. The examiner should address test security requirements with the examinee (and facilitator, if applicable) during the informed consent process. The examiner should make it clear that the video should not be captured, photos should not be taken, and stimuli should not be copied or recorded, because these actions are copyright violations. The examinee must agree that they will not record (audio or visual) or take photos or screenshots of any portion of the test materials or testing session, and will not permit anyone to observe the testing session or be in the testing room (except for a facilitator, when necessary). Any response booklets used in the testing session must be returned to the examiner (see Assessment Procedures & Materials for more information).

Peripheral camera or device

A stand-alone peripheral camera that can be positioned to provide a view of the session from another angle or a live view of the examinee’s progress is helpful. Alternately, a separate device (e.g., a smartphone with a camera or another peripheral device) can be connected to the teleconference and set in a stable position to show the examinee’s pointing or written responses. The device’s audio should be silenced, the microphone should be muted to prevent feedback, and smartphones should be set in airplane mode with WiFi access turned off. The examiner should guide positioning of the peripheral camera/device before administering written response tasks (i.e., Trail Making, Design Fluency) so that the examiner can see that the examinee’s real-time responses. Corrective feedback is provided during performance so this is critical for administration. A professional facilitator, trained to detect and intervene when errors are made, may be needed to ensure appropriate administration.

In an ideal telehealth session, a document or moveable camera would be available in the examinee’s location. However, while social distancing is necessary, the only camera available may be a stationary camera integrated into the examinee’s laptop or computer screen. It is unrealistic to expect examinees to have specialized cameras within their homes that will show their responses as they take the tests (e.g., their drawings on the Trail Making Test record forms). It may be necessary for examiners to think creatively about how to use a smartphone in the examinee’s location to gain a view of the examinee’s progress in a response booklet or when pointing at a screen. Before attempting this with an examinee, the examiner should work to become fluid and competent at directing examinees in these methods, which can require extensive practice with varied individuals and types of smartphones. In addition, this requires planning and practice in the initial virtual meeting to prevent technical difficulties and to help the examinee feel confident doing this when it is time.

Online instructional videos (e.g., here) demonstrate how a smartphone may be used with common household objects (e.g., a tower or stack of books, paper weight, ruler, and rubber band or tape) to create an improvised document camera for use during tasks involving response booklets. Similarly, on some tests (e.g., Color–Word Interference Test), some examinees may point to test stimuli as they respond to the items. In this situation, other everyday household objects (e.g., books) could be used to form an improvised stand upon which to position a device to provide a second-angle view of the examinee pointing at the screen. Typically, devices provide the best view of the examinee’s screen and pointing responses when positioned in landscape format. While using a smartphone as the peripheral camera is not an optimal solution for telehealth, it can be functional if executed well.

Gesturing

Several D–KEFS tests require that the examiner point or gesture to the stimulus image or response booklet throughout administration. For Color–Word Interference and Tower, the examiner may display the digital assets onscreen and point using the mouse cursor for pointing. Other tests may require a professional facilitator to provide gestures as needed. Refer to Table 1 for specific instructions by subtest.

Capturing response booklet performance (if used)

The Trail Making Test and Design Fluency Test use response booklets and will therefore usually require administration with a trained facilitator because of the complex interplay of multiple response booklets, teaching and practice, timing, and immediate corrective feedback during test conditions. (The exception to this may be individuals with higher levels of cognitive skills.) After completing the administration of the different conditions of each of these tests, the examiner may ask for the completed response booklets to be shown on camera so that they can take a screenshot of it for later scoring (which ensures that the responses are not lost or modified). At the conclusion of each test, the record forms should be placed and sealed in a return envelope by the facilitator or examinee and mailed to the examiner.

Audio considerations

High-quality audio capabilities are required during the administration. An over-the-head, two-ear, stereo headset with attached boom microphone is recommended for both the examiner and examinee. Headphones with a microphone may be used if a headset is not available.

Audio check

The examiner should test the audio for both the examiner and examinee in the initial virtual meeting and at the beginning of the testing session to ensure a high-quality audio environment is present. This is especially important for Verbal Fluency, Color–Word Interference, Twenty Questions, Word Context, and Proverbs. Testing the audio should include an informal conversation prior to the administration where the examiner is listening for any clicks, pops, or breaks in the audio signal that distorts or interrupts the voice of the examinee. The examiner should also ask if there are any interruptions or distortions in the audio signal on the examinee’s end. Any connectivity lapses, distractions, or intrusions that occurred during testing should be reported.

Manage audiovisual distractions: As with any testing session, the examiner should do everything possible to make sure the examinee’s environment is free from audio and visual distractions. If the examiner is unfamiliar with the examinee’s planned physical location, a visual tour of the intended testing room should be given during the initial virtual meeting. The examiner can then provide a list of issues to address to transform the environment into one suitable for testing. For example, remove distracting items, silence all electronics, and close doors. The examiner should confirm that these issues have been addressed at the time of testing. If possible, the examinee should be positioned facing away from the door to ensure the examiner can verify through the examinee’s camera that the door remains shut and can monitor any interruptions. The examiner should confirm that all other applications on the computer, laptop, or peripheral device are closed, the keyboard is moved aside or covered after the session is connected, and alerts and notifications are silenced on the peripheral device. Radios, televisions, other cellular phones, fax machines, smart speakers, printers, and equipment that emit noise must be silenced and/or removed from the room. In most cases, pets should also not be in the testing room.

Lighting

Good overhead and facial lighting should be established for the examiner and examinee. Blinds or shades should be closed to reduce sun glare on faces and the computer screens.

Disruptions

The examiner should record any and all atypical events that occur during the testing session. This may include delayed audio or video, disruptions to connectivity, the examinee being distracted by external stimuli, and any other anomalies (e.g., dogs barking, children crying). These can be noted on the record form or in the Q-interactive notes and should be considered during interpretation and described in the written report.

2. Assessment Procedures & Materials

Copyright

Permission must be obtained for access to copyrighted materials (e.g., stimulus books, response booklets) as appropriate. Pearson has provided a letter of No Objection to permit use of copyrighted materials for telehealth via teleconferencing software and tools to assist in remote administration of assessment content during the COVID-19 pandemic.

Response booklets (if used)

The response booklets should be provided in advance of the testing session, and the plan for securing and forwarding/returning materials—real-time and after testing—should be communicated. See the Capturing response booklet performance portion of the Telehealth Environment & Equipment section for suggested procedures to facilitate immediate scoring and secure handling of response booklets. Ensure that the correct response booklet is presented for each condition of the Trail Making and Design Fluency tests because the response booklets for the different conditions look similar.

Digital assets

The examiner should practice using the digital assets until the use of the materials is as smooth as a face-to-face administration. It is not recommended that the examiner hold up the stimulus book to display items on a camera during administration.

Considerations

Review Table 1 for the specific telehealth considerations for each subtest to be administered.

Input and output requirements and equivalence evidence

The examiner should consider the input and output requirements for each task, and the evidence available for telehealth equivalence for the specific task type.

Telehealth Versus Face-to-Face Administration

Preliminary research has compared results obtained in telehealth and face-to-face administration modes. One study (Harder et al., 2020) used the D–KEFS Verbal Fluency and Color–Word Interference tasks remotely with children ages 10–17 years. The visual stimuli were sent to the examinee’s home in a sealed envelope and were opened in front of the examiner during administration. Examinees completed all testing in their home. Upon completion of the testing session, the materials were sealed in an envelope and returned to the examiner. In this study, no difference was found between scores obtained through the remote testing and those who were tested in face-to-face sessions. Several tasks with similar administration and response requirements to those found in the D–KEFS have produced evidence of equivalence in telehealth and face-to-face modes for examinees with a variety of clinical conditions (Cullum et al., 2006, 2014; Dekhytar et al., 2020; Galusha-Glasscock et al., 2016; Hildebrand et al., 2004; Hodge et al., 2019; Jacobsen, 2003; Ragbeer et al., 2016; Stain et al., 2011; Vahia et al., 2015; Vestal et al., 2006; Wadsworth et al., 2018). Other studies support equivalence of tasks that are similar to some of the D–KEFS tests with nonclinical examinees using telehealth compared with face-to-face administration and scoring (Galusha-Glasscock et al., 2016; Wright, 2020). In addition, a meta-analysis of telehealth studies provides support for telehealth and face-to-face mode equivalence across a variety of neuropsychological tests (Brearly et al., 2017).

While equivalence data on similar measures are relevant, practitioners should be mindful that more research is needed to establish equivalence in all ages and for all tasks on the D–KEFS. Additional caveats and cautions are described in Grosch et al. (2011). Also, most telehealth-based studies were conducted with volunteer subjects in controlled environments. When social distancing is critical (such as during the COVID-19 pandemic), some examinations may need to occur in patients’ homes, and it should be noted that very little research has been done about remote assessment in private homes.

It is important to consider the conditions under which equivalence studies of telehealth and face-to-face assessment modes were conducted and to attempt to reproduce these as closely as possible if testing via telehealth. Typical telehealth studies that support telehealth and face-to-face equivalence involve the examiner becoming very familiar with the teleconference platform by using it for its intended purpose for several hours and administering tests (even those that are familiar in face-to-face mode) multiple times to “practice examinees.” Some studies that have established telehealth and face-to-face mode equivalence involve a professional facilitator. However, preliminary research conducted and described by Lana Harder (Stolwyk et al., 2020) with parents serving as in-home facilitators who managed audiovisual needs and response booklets found no significant differences across modes. Finally, the examinee is typically in an office- or school-based setting. Therefore, if in-home assessment is taking place, it is advisable to prepare a similar environment as much as possible as described in the Telehealth Environment & Equipment section.

Digital Versus Traditional Format

Telehealth involves the use of technology in assessment as well as viewing onscreen stimuli. For these reasons, studies that investigate assessment in digital versus traditional formats are also relevant.

During the development of the D–KEFS on Q-interactive, an equivalency study was conducted on Design Fluency, Verbal Fluency, Trail Making, and Color–Word Interference. It is important to note that for these subtests all the materials used for paper administration were retained, with the exception of the display of written rules for the Verbal Fluency subtest. The paper response booklets for Trail Making and Design Fluency were retained and the stimulus book was used for Color–Word Interference. The other tests were not provided in Q-interactive. The Q-interactive platform provided all the instructions, response capture, and scoring for the examiner. Daniel (2012) found no difference in raw scores on the measures between the paper and digital administrations. It is important to note that these studies were not conducted remotely or via video conference.

Evidence by Subtest

Table 2 lists each D–KEFS test, the input and output requirements, and the evidence for tasks with similar task requirements.

3. Examinee Considerations

Appropriateness

The examiner should first ensure that a telehealth administration is appropriate for the examinee and for the purpose of the assessment. Clinical judgment, best practice guidance for telehealth (e.g., APA Services, 2020; ASPPB, 2013; IOPC, 2020), information from professional organizations and other professional entities (e.g., licensing boards, legal resources, professional liability insurance providers, payors), consultation with other knowledgeable psychologists, existing research, and any available federal or state regulations should be considered in the decision-making process. Consideration should be given to whether the necessary administrative and technological tasks involved in a telehealth session can be accomplished without influencing results.

Preparedness

Before initiating test administration, the examiner should ensure that the examinee is well-rested, able, prepared, and ready to appropriately and fully participate in the testing session.

Facilitator role

If using a facilitator, the role of the facilitator must be explained to the examinee so participation and actions are understood.

Headset

It may not be appropriate or feasible for some examinees to use a headset due to behavior, positioning, physical needs, or tactile sensitivities, or if a headset is not available. Clinical judgment on the appropriate use of a headset in these situations should be used. If a headset is not used, the examiner’s and examinee’s microphones and speakers should be turned up to a comfortable volume.

Mouse

On some teleconference platforms, the examiner can pass control of the mouse to allow the examinee to point to indicate responses; this is an option if it is within the capabilities of the examinee. However, best practice guidelines provide cautions about this. For example, the IOPC guidelines suggest examiners be alert throughout administration, return control of the screen once the task is finished, and never leave the computer unattended while the examinee has control over the examiner’s computer (IOPC, 2020).

4. Examiner Considerations

Practice

During the telehealth setup, and before administering to any actual examinee, the examiner should rehearse the mechanics and workflow of every item in the entire test using the selected teleconference platform so that the examiner is familiar with the administration procedures. For example, a colleague could be used as a practice examinee.

Standardized procedures

The examiner must follow the administration procedures of face-to-face administration as much as possible. For example, if a spoken stimulus cannot be said more than once in face-to-face administration, the examiner must not say it more than once in a telehealth administration unless a technical difficulty precluded the examinee from hearing the stimulus.

Administrative and technological tasks

In order to conduct a smooth telehealth session, audiovisual needs and materials must be managed appropriately. The initial virtual meeting involves the examiner, examinee, and/or the facilitator (if used), and presents the opportunity for the examiner to provide information about the audiovisual needs and materials. During the initial virtual meeting, the examiner should provide training in troubleshooting audiovisual needs that arise during the testing session, including camera angle, lighting, and audio checks. The examiner should provide verbal feedback to guide camera adjustment, checking the onscreen video shown by the peripheral camera/device to provide information about how to reposition it until the proper view is shown. The examiner should emphasize that no materials should be opened until the examiner provides instructions to do so, if applicable. The examiner should also expect to provide verbal guidance about these issues during the testing session. Refer to the Telehealth Environment & Equipment section and to Table 1 for specific subtest telehealth considerations.

If used, the facilitator should assist with administrative and technological tasks but not manage rapport, engagement, or attention during the testing session. The examiner should direct facilitators not to interfere with the examinee’s performance or responses. Any other roles and responsibilities for which an examiner needs support, such as behavior management, should be outlined and trained prior to the beginning of the testing session. The examiner is responsible for documenting the technical efficiency with which the exam occurred and all behaviors of the facilitator during test administration, and should take these into consideration when reporting scores and performance.

5. Other Considerations

There are special considerations for written reports describing testing that takes place via telehealth. The professional completing the written report should state in the report that the test was administered via telehealth and briefly describe the method of telehealth used. For example, “The D–KEFS Twenty Questions test was administered via remote telehealth using digital stimulus materials on Pearson’s Q-global system, and a facilitator monitored the administration onsite during the live video connection using the [name of telehealth system, e.g., Zoom] platform.”

The professional should also make a clinical judgment, similar to a face-to-face session, about whether or not the examiner was able to obtain the examinee’s best performance. Clinical decisions should be explained in the report, including comments on the factors that led to the decision to conduct testing via telehealth and to report all (or not to report suspect) scores. In addition, it is recommended that the report include a record of any and all atypical events during the testing session (e.g., delayed video or audio, disruptions to connectivity, extraneous noises such as phone ringing or loud dog barking, person or animal unexpectedly walking into room, the examinee responding to other external stimuli). Notes may be recorded about these issues on the record form or in the notes section on Q-interactive. List and describe these anomalies as you would typically for reporting behavioral observations in the written report, as well as any observed or perceived impact on the testing sessions and/or results, and consider these in the interpretation of results. For example, “The remote testing environment appeared free of distractions, adequate rapport was established with the examinee via video/audio, and the examinee appeared appropriately engaged in the task throughout the session. No significant technological problems or distractions were noted during administration. Modifications to the standardization procedure included: [list]. Tasks with similar administration and response demands to those on the D–KEFS have received initial validation in several samples for remote telehealth and digital format administration, and the results are considered a valid description of the examinee’s skills and abilities.”

Conclusion

The D–KEFS was not standardized in a telehealth mode, and this should be taken into consideration when using this test via telehealth and interpreting results. For example, the examiner should consider relying on convergence of multiple data sources and/or being tentative about conclusions. Provided that the examiner has thoroughly considered and addressed the factors and the specific considerations as listed above, the examiner should be prepared to observe and comment about the reliable and valid delivery of the test via telehealth. Materials may be used via telehealth without additional permission from Pearson in the following published contexts:

D–KEFS manual, digital stimulus book, and response booklets via Q-global
D–KEFS via Q-interactive (requires advanced technology skills and mirroring software)

Any other use of the D–KEFS via telehealth is not currently recommended. This includes, but is not limited to, scanning the paper stimulus books, digitizing the paper record forms, holding the stimulus books physically up in the camera’s viewing area, or uploading a manual onto a shared drive or site.

References

American Psychological Association Services (APA Services). (2020). Guidance on psychological tele-assessment during the COVID-19 crisis. (2020). https://www.apaservices.org/practice/reimbursement/health-codes/testing/tele-assessment-covid-19?fbclid=IwAR1d_YNXYS2Yc5mdIz_ZIYSkrrJ_6A9BQeKuIHxEEjjRh1XDR6fOYncM3b4

Brearly, T., Shura, R., Martindale, S., Lazowski, R., Luxton, D., Shenal, B., & Rowland, J. (2017). Neuropsychological test administration by videoconference: A systematic review and meta-analysis. Neuropsychology Review, 27(2), 174–186. https://doi.org/10.1007/s11065-017-9349-1

Cullum, C. M., Hynan, L. S., Grosch, M., Parikh, M., & Weiner, M. F. (2014). Teleneuropsychology: Evidence for video teleconference-based neuropsychological assessment. Journal of the International Neuropsychological Society, 20(10), 1028–1033. https://doi.org/10.1017/S1355617714000873

Cullum, C. M., Weiner, M. F., Gehrmann, H. R., & Hynan, L. S. (2006). Feasibility of telecognitive assessment in dementia. Assessment, 13(4), 385–390. https://doi.org/10.1177/1073191106289065

Daniel, M. H. (2012). Equivalence of Q-interactive administered cognitive tasks: CVLT-II and Selected D–KEFS subtests (Q-interactive Technical Report 3). Pearson. https://www.pearsonassessments.com/content/dam/school/global/clinical/us/assets/q-interactive/005-s-Technical%20Report%203_CVLT_DKEFS_final_rev.pdf

Dekhtyar, M., Braun, E. J., Billot, A., Foo, L., & Kiran, S. (2020). Videoconference administration of the Western Aphasia Battery–Revised: Feasibility and validity. American Journal of Speech-Language Pathology, 29(2), 673–687. https://doi.org/10.1044/2019_AJSLP-19-00023

Delis, D. C., Kaplan, E., & Krammer, J. H. (2001). Delis-Kaplan Executive Function System. San Antonio: Pearson.

Eichstadt, T. J., Castilleja, N., Jakubowitz, M., & Wallace, A. (2013, November). Standardized assessment via telehealth: Qualitative review and survey data [Paper presentation]. Annual meeting of the American-Speech-Language-Hearing Association, Chicago, IL, United States.

Galusha-Glasscock, J. M., Horton, D. K., Weiner, M. F., & Cullum, C. M. (2016). Video teleconference administration of the Repeatable Battery for the Assessment of Neuropsychological Status. Archives of Clinical Neuropsychology, 31(1), 8–11. https://doi.org/10.1093/arclin/acv058

Grosch, M. C., Gottlieb, M. C., & Cullum, C. M. (2011). Initial practice recommendations for teleneuropsychology. The Clinical Neuropsychologist, 25(7), 1119–1133. https://doi.org/10.1080/13854046.2011.609840

Harder, L., Hernandez, A., Hague, C., Neumann, J., McCreary, M., Cullum, C. M., & Greenberg, B. (2020). Home-based pediatric teleneuropsychology: A validation study. Archives of Clinical Neuropsychology, 35(8), 1266–1275. https://doi.org/10.1093/arclin/acaa070

Hildebrand, R., Chow, H., Williams, C., Nelson, M., & Wass, P. (2004). Feasibility of neuropsychological testing of older adults via videoconference: Implications for assessing the capacity for independent living. Journal of Telemedicine and Telecare, 10(3), 130–134. https://doi.org/10.1258/135763304323070751

Hodge, M., Sutherland, R., Jeng, K., Bale, G., Batta, P., Cambridge, A., Detheridge, J., Drevensek, S., Edwards, L., Everett, M., Ganesalingam, K., Geier, P., Kass, C., Mathieson, S., McCabe, M., Micallef, K., Molomby, K., Ong, N., Pfeiffer, S., … Silove, N. (2019). Agreement betweentelehealth and face-to-face assessment of intellectual ability in children with specific learning disorder. Journal of Telemedicine and Telecare, 25(7), 431–437. https://doi.org/10.1177/1357633X18776095

Inter Organizational Practice Committee (IOPC). (2020). Recommendations/guidance for teleneuropsychology (TeleNP) in response to the COVID-19 pandemic. https://static1.squarespace.com/static/50a3e393e4b07025e1a4f0d0/t/5e8260be9a64587cfd3a9832/1585602750557/Recommendations-Guidance+for+Teleneuropsychology-COVID-19-4.pdf

Jacobsen, S. E., Sprenger, T., Andersson, S., & Krogstad, J.-M. (2003). Neuropsychological assessment and telemedicine: A preliminary study examining the reliability of neuropsychology services performed via telecommunication. Journal of the International Neuropsychological Society, 9(3), 472–478. https://doi.org/10.1017/S1355617703930128

Ragbeer, S. N., Augustine, E. F., Mink, J. W., Thatcher, A. R., Vierhile, A. E., & Adams, H. R. (2016). Remote assessment of cognitive function in juvenile neuronal ceroid lipofuscinosis (Batten disease): A pilot study of feasibility and reliability. Journal of Child Neurology, 31(4), 481–487. https://doi.org/10.1177/0883073815600863

Stain, H. J., Payne, K., Thienel, R., Michie, P., Carr, V., & Kelly, B. (2011). The feasibility of videoconferencing for neuropsychological assessments of rural youth experiencing early psychosis. Journal of Telemedicine and Telecare, 17(6), 328–331. https://doi.org/10.1258/jtt.2011.101015

Stolwyk, R., Hammers, D. B., Harder, L., & Cullum, C. M. (2020). Teleneuropsychology (TeleNP) in response to COVID-19.

Temple, V., Drummond, C., Valiquette, S., & Jozsvai, E. (2010). A comparison of intellectual assessments over video conferencing and in-person for individuals with ID: Preliminary data. Journal of Intellectual Disability Research, 54(6), 573–577. https://doi.org/10.1111/j.1365-2788.2010.01282.x

Turkstra, L. S., Quinn-Padron, M., Johnson, J. E., Workinger, M. S., & Antoniotti, N. (2012). In-person versus telehealth assessment of discourse ability in adults with traumatic brain injury. The Journal of Head Trauma Rehabilitation, 27(6), 424–432. https://doi.org/10.1097/HTR.0b013e31823346fc

Vahia, I. V., Ng, B., Camacho, A., Cardenas, V., Cherner, M., Depp, C. A., Palmer, B. W., Jeste, D. V., & Agha, Z. (2015). Telepsychiatry for neurocognitive testing in older rural Latino adults. American Journal of Geriatric Psychiatry, 23(7), 666–670. https://doi.org/10.1016/j.jagp.2014.08.006

Vestal, L., Smith-Olinde, L., Hicks, G., Hutton, T., & Hart, J., Jr. (2006). Efficacy of language assessment in Alzheimer’s disease: Comparing in-person examination and telemedicine. Clinical Interventions in Aging, 1(4), 467–471. https://doi.org/10.2147/ciia.2006.1.4.467

Wadsworth, H. E., Dhima, K., Womack, K. B., Hart Jr, J., Weiner, M. F., Hynan, L. S., & Cullum, C. M. (2018). Validity of teleneuropsychological assessment in older patients with cognitive disorders. Archives of Clinical Neuropsychology, 33(8), 1040-1045.

Waite, M. C., Theodoros, D. G., Russell, T. G., & Cahill, L. M. (2010). Internet-based telehealth assessment of language with CELF-4. Language, Speech, and Hearing Services in Schools, 41(4), 445–458.

Wright, A.J. (2018a). Equivalence of remote, online administration and traditional, face-to-face administration of the Woodcock-Johnson IV cognitive and achievement tests. Archives of Assessment Psychology, 8(1), 23-35.

Wright, A. J. (2018b). Equivalence of remote, online administration and traditional, face-to-face administration of the Reynolds Intellectual Assessment Scales-Second Edition. https://pages.presencelearning.com/rs/845-NEW-442/images/Content-PresenceLearning-Equivalence-of-Remote-Online-Administration-of-RIAS-2-White-Paper.pdf

Wright, A. J. (2020). Equivalence of remote, digital administration and traditional, in-person administration of the Wechsler Intelligence Scale for Children, Fifth Edition (WISC-V). Psychological Assessment, 32(9), 809–817. https://doi.org/ 10.1037/pas0000939

Assessments by category

Assessments by profession

Digital solutions

Featured topics

Training & demonstrations

Webinars

Your research

Join our research

Ordering

Manage orders

Account information

Official distributor

Contact us

Telehealth and the D-KEFS

Conducting Telehealth Assessment

Conclusion

References

A customer reflects on using Q-global digital stimulus books and manuals:

ASSESSMENTS

CLINICAL LEGAL POLICIES

HELP & SUPPORT

ABOUT PEARSON