Field Evaluation of Roadside Test Devices

3.4.1. Study Methods

In this part of the study, 11 on-site urine-test devices, 3 oral-fluid test devices, and 1 sweat-test device were tested in 2968 drivers, and positive and negative results were compared with those obtained by the reference methods as well as those in blood samples assessed by the same reference methods (6). The reference methods referred to in this study are mostly gas chromatogra-phy(GC)-mass spectrometry (MS) or, in some cases, high-performance liquid chromatography with diode array detection (HPLC-DAD) or GC with electron capture detection (GC-ECD) (see Chapter 3). The number of subjects involved in each category and the drugs tested are given in Table 2.

Because of different legislation, the circumstances under which the tests were performed varied among the countries:

• Belgium: samples collected at the roadside were first screened by police with the Dipro Drugscreen 5 (Dipro Diagnostics) device and then by lab technicians with the other devices.

• Finland: urine was collected under police supervision in the hospital and not at the roadside. Police and laboratory staff performed the urine tests at the laboratory. Oral-fluid tests were performed at the roadside by trained police officers.

• France: on-site tests were evaluated in the laboratory.

• Germany: the tests were performed by police officers during police patrols. Oral-fluid and sweat samples were collected and tested directly at the roadside, whereas urine samples were normally collected and tested at police stations or at public lavatories. Patrols were conducted during the night, rendering reading of the results more difficult than in a police station, hospital, or laboratory.

• Italy: on-site tests were performed at the roadside by police personnel or ambulance volunteers, or in the laboratory by trained technicians. Roadside collection of blood, urine, and oral-fluid samples were made by medical personnel.

• Norway: on-site urine tests were performed by the police officers in the laboratory at the National Institute for Forensic Toxicology, in collaboration with

Table 2

Overview of the Methodological Aspects of the Roadside Testing Assessment (ROSITA) Field Tests

Table 2

Overview of the Methodological Aspects of the Roadside Testing Assessment (ROSITA) Field Tests

Countries

Number of subjects

Specimens collected

Number of on-site test devices evaluated in urine/oral fluid/sweat

Place of tests

Personnel performing tests

Belgium

180

Blood,

urine, oral fluid, sweat

4/1/1

Roadside

Police, laboratory personnel

Finland

751

Blood,

urine, oral fluid

11/2

Lab

Police, laboratory personnel

France

198

Blood,

urine, oral fluid, sweat

3/0

Hospital

Laboratory personnel

Germany

617

Blood,

urine, oral fluid, sweat

6/2/1

Roadside/

Police, laboratory personnel

police station

Italy

302

Blood,

urine, oral fluid

3/1

Roadside/lab

Police, Red Cross volunteers

Norway

314

Blood,

urine, oral fluid

3/2

Lab/station

Police

Spain

384

Urine,

oral fluid

3/2

Roadside

Police, laboratory personnel

United Kingdom

214

Blood,

urine, oral fluid

3/1

Prison/lab

Laboratory personnel

representatives from some manufacturers as assistants. The oral-fluid tests (Cozart Rapiscan [Cozart Bioscience Ltd.] and Drugwipe® [Securetec Detektions-Systeme AG]) were performed at the police station.

• Spain: on-site tests were performed by agents of the traffic police. Reading and interpretation of the results were done together by members of the Institute of Legal Medicine present during the patrol and by traffic police officers trained in the use of the devices. With one exception, the tests were performed at the roadside.

• United Kingdom: the subjects were prisoners. The on-site tests were performed by at least two members of the research team, either in the prison or in the laboratory.

The data from the evaluations in the eight countries were displayed in Microsoft Excel format. For the evaluation of opiates, specimens containing morphine, 6-acetylmorphine, or codeine were considered positives. It should be noted that other substances may cross-react and give positive results with on-site tests—for example, dihydrocodeine or pholcodine. For determination of the optimal cut-off levels in oral fluid, receiver operating characteristic (ROC) curves (7)) were used.

Several comparisons were made between the different methods (on-site tests or reference methods) and the different matrices (blood, urine, oral fluid, or sweat). For each drug class, the following comparisons were made:

• A comparison between the reference method for blood and for the other biological fluids was made in order to assess whether findings in each matrix correspond well to those in blood. There is a general consensus that blood is the standard reference, as impairment (or recent exposure to drugs) corresponds best to the presence of drugs in blood;

• A comparison of on-site device results with those obtained from the reference methods for the same matrix;

• The validity of the roadside test for predicting blood positives by comparison with the results of blood samples assessed with a reference method.

For evaluation of the urine, oral-fluid, or sweat test devices, the accuracy, sensitivity, and specificity values have been calculated on the basis of the following definitions, where TP = true-positives, TN = true-negatives, FP = false-positives, and FN = false-negatives:

• Accuracy = percent of all samples correctly identified by the tests = (TP + TN)/ (all results)

• Sensitivity = true-positives expressed as percent of all positives = TP/(TP + FN)

• Specificity = true-negatives expressed as percent of all negatives = TN/(TN + FP)

The following analytical criteria for an acceptable test were used: accuracy >95%, sensitivity >90%, specificity >90%, when compared with a reference method. Statistical analysis of the data was performed using Microsoft Excel, Medcalc (MedCalc Software), and SPSS (SPSS, Inc.).

3.4.2. Results

The study was performed on 2968 subjects. Ninety-two percent were males. Before evaluating the performance of on-site test devices, positive and negative results of a given drug class in oral-fluid, sweat, and urine samples assessed by a reference method (e.g., GC-MS) were compared with the results of blood samples assessed by the same reference method. This process allowed for the computation of the sensitivity, specificity, and accuracy of using a given reference method (e.g., GC-MS) in different fluids. In general, one can conclude that the accuracy rate of such comparisons using GC-MS ranges between 78 (cannabinoids in sweat) and 99% (cocaine in oral fluid) (Table 3). The exception is the benzodiazepine group, where the low sensitivity of the analytical methods applied in the laboratory appears to be the cause for the overall low and insufficient accuracy rate of oral fluid (29%) as a specimen for roadside screening. It is clear that there will never be a 100% correlation among the different body fluids. The results are influenced by the timing of sampling relative to the last drug intake. If a drug was taken very recently, it is possible that it can be found only in blood (and oral fluid), but not yet in urine. If a drug was taken a longer time ago, it is possible that it is no longer detectable in blood, but only in urine (and possibly in sweat).

In the next set of experiments, drug results from oral-fluid, sweat, and urine samples as assessed by various on-site testing devices were compared with those of the same fluid assessed by the reference method (i.e., GC-MS) or those of blood samples assessed by the same reference method. These results are given in Tables 4-6. The following sections provide a summary of the findings.

For amphetamines, all body fluids were appropriate for testing of this drug when GC-MS was the reference method. Both urine and oral fluid have good accuracy and predictive values (Table 3). Eighteen different on-site tests for amphetamine or methamphetamine were evaluated. If the results of amphetamines and methamphetamine were considered jointly (i.e., if one considers the test to be positive if either the amphetamine or the methamphetamine test is positive), test devices such as Rapid Drug Screen® (RDS; American Bio Medica), Dipro, and Syva® RapidTest™ (SRT; Dade Behring) satisfied the analytical criteria (accuracy >95%, sensitivity >90%, specificity >90%) (Table 4). Tests for oral fluid had much lower accuracy (80% or less in all cases; Table 5). The optimal cut-off for amphetamines in oral fluid was in the range of 70-90 ng/mL. For sweat, the low number of samples (nearly all positive) did not permit definite conclusions (Table 6), but use of sweat seemed promising.

Urine seemed to be a better fluid for detecting benzodiazepines at the roadside with the reference methods used (Table 3). Of the on-site urine test devices, Triage (Biosite Diagnostics) and RDS were the only two that met the

Table 3

Comparison of the Accuracy, Sensitivity, and Specificity of the Qualitative Results by Gas Chromatography (GC)-Mass Spectrometry (MS) in Urine, Oral Fluid, and Sweat vs GC-MS in Blood ("Gold Standard") for the Different Drugs

Table 3

Comparison of the Accuracy, Sensitivity, and Specificity of the Qualitative Results by Gas Chromatography (GC)-Mass Spectrometry (MS) in Urine, Oral Fluid, and Sweat vs GC-MS in Blood ("Gold Standard") for the Different Drugs

Analyte

Accuracy

Sensitivity

Specificity

Urine

Oral fluid

Sweat

Urine

Oral fluid

Sweat

Urine

Oral fluid

Sweat

Amphetamine

94%

95%

97%

97%

98%

100%

92%

91%

0%

Benzodiazepines

89%

29%

89%

21%

90%

67%

Cannabinoids

86%

89%

78%

97%

86%

91%

81%

90%

17%

Cocaine

97%

99%

89%

95%

96%

100%

98%

99%

0%

Opiates

86%

91%

80%

97%

89%

88%

85%

91%

63%

Number of Comparisons (/i), Sensitivity (Se, %), Specificity (Sp, %) and Accuracy (Ac, %) of Rapid Urine Tests for Five Drug Classes

Amphetamine +

methamphetamine Benzodiazepines Cannabis Cocaine Opiates

n Se Sp Ac n Se Sp Ac n Se Sp Ac n Se Sp Ac N Se Sp Ac

American Biomedica

Ah

- M

468

98

99

99

219

91

98

97

571

97

90

92

580

100

98

98

472

98

95

95

Rapid Drag

Screen®

Cortez

Ah

- M

186

87

93

90

189

81

84

82

369

95

95

95

393

85

98

97

387

98

95

95

Dipro Dragscreen 5

panel test

Ah

- M

122

97

100

98

123

99

92

97

128

100

99

99

34

100

85

88

Frontline

A

68

0

56

68

Mahsan Diagnostica

A

157

88

99

97

148

97

91

94

156

100

93

94

137

-

97

97

Rapitest® Multidrug

A

95

86

96

92

92

95

82

91

95

70

98

85

96

75

100

99

97

78

99

97

panel

Roche TestCup 5

A

527

75

100

95

542

92

93

93

570

95

99

99

474

97

93

94

Status DS

A

92

85

96

91

92

80

100

91

92

100

99

99

94

100

97

97

Surescreen 6 Drag

MultiTest

Ah

- M

106

93

95

94

102

89

88

88

114

76

99

90

116

100

100

100

118

82

97

96

Syva® RapidCup™

10

A

52

0

100

100

88

97

92

94

90

100

98

98

85

100

96

96

Syva RapidTest™

Ah

- M

558

97

100

100

354

98

84

86

880

93

100

97

904

96

99

99

782

95

96

96

Triage

A

395

89

99

98

394

94

99

98

396

84

99

96

396

95

100

100

396

100

99

99

The results were compared with the results obtained by gas chromatography-mass spectrometry. For amphetamines, in some cases (A + M) the combination of an amphetamine and methamphetamine test was used (see text).

Table 5

Accuracy, Sensitivity, and Specificity Values With Respect to the Blood Status Measured With Gas Chromatography-Mass Spectrometry for Three Oral Fluid Test Kits

Tests Drugwipe® Rapiscan ORALscreen™

Table 5

Accuracy, Sensitivity, and Specificity Values With Respect to the Blood Status Measured With Gas Chromatography-Mass Spectrometry for Three Oral Fluid Test Kits

Tests Drugwipe® Rapiscan ORALscreen™

Acc.

Sens.

Spec.

Acc.

Sens.

Spec.

Acc.

Sens.

Spec.

Type of Drag

N

(%)

(%)

(%)

N

(%)

(%)

(%)

N

(%)

(%)

(%)

Amphetamine

142

73

90

55

111

80

87

74

0

MDMA ("ecstasy")

130

72

90

55

61

74

67

74

0

Benzodiazepines

0

133

56

17

90

0

Cannabinoids

0

98

79

16

94

179

84

50

84

Cocaine

34

82

75

93

4

50

50

50

190

99

99

Opiates

214

81

63

83

109

81

67

83

180

86

50

87

MDMA, methylenedioxymethamphetamine.

MDMA, methylenedioxymethamphetamine.

Table 6

Accuracy, Sensitivity, and Specificity Values for Drugwipe® With Respect to the Blood and Sweat Status Measured With Gas Chromatography (GC)-Mass Spectrometry (MS)

Tests Drugwipe

Sweat vs blood GC-MS Sweat vs sweat GC-MS

Table 6

Accuracy, Sensitivity, and Specificity Values for Drugwipe® With Respect to the Blood and Sweat Status Measured With Gas Chromatography (GC)-Mass Spectrometry (MS)

Sweat vs blood GC-MS Sweat vs sweat GC-MS

Acc.

Sens.

Spec.

Acc.

Sens.

Spec.

Type of drug

n

(%)

(%)

(%)

n

(%)

(%)

(%)

Amphetamine

38

95

100

0

37

92

94

67

MDMA ("ecstasy")

59

97

100

0

54

96

98

67

Cocaine

22

68

75

0

22

77

77

NA

Opiates

12

83

100

50

9

89

89

NA

NA, not applicable or not available; MDMA, metliylenedioxymethamphetamine.

NA, not applicable or not available; MDMA, metliylenedioxymethamphetamine.

analytical criteria (Table 4). For oral fluid, the sensitivity of the on-site test devices and of some confirmation methods was very poor (Table 5). This was explained by the extremely low concentrations of benzodiazepines in oral fluid (often <1 ng/mL). This was even more so for the low levels of some benzo-diazepines, such as flunitrazepam. No on-site tests were available for sweat.

For cannabinoids, comparison of the performance of the different matrices showed a small advantage for oral fluid (89% accuracy; Table 3), which is not unexpected considering the much longer window of detection of cannabis metabolites in urine compared with the presence of tetrahydrocannabinol (THC) in blood. Three of 11 on-site tests for urine met the analytical criteria. These were Dipro, Cortez (Cortez Diagnostics), and SRT (Table 4). In comparison with blood, the accuracy of the best on-site urine tests was close to 90%. For the on-site oral-fluid tests (Table 5), the sensitivity was too low (only 18 to 25% when compared with blood results). The required sensitivity of on-site oral-fluid tests was 2 ng/mL of THC. There were indications that THC may bind to the material of some sampling devices. Much higher concentrations of THC could be extracted from the cotton of the Salivette®, in comparison with the THC concentrations in oral fluid. A possible explanation could be that the cotton of the Salivette absorbs the THC that has been sequestered onto teeth and gum, but this possibility needs to be confirmed. This phenomenon could be useful for increasing the sensitivity of oral-fluid analysis for THC, if a suitable extraction method can be found to release the THC trapped on the fibers of the sampling device. For testing of cannabinoids with sweat, no on-site test devices were available.

For cocaine and metabolites, both oral fluid and urine gave good correlation for the prediction of positive drug results in blood assessed by GC-MS (Table 3). Eight of the 11 on-site tests met the analytical criteria. These were Dipro, RDS, TesTcup (Roche Diagnostics.), Syva Rapid Cup (SRC), SRT, SureScreen™ (Surescreen Diagnostics), Status DS™ (Lifesign), and Triage (Table 4). Even when compared with blood results, four tests had an accuracy of greater than 95% and sensitivity and specificity greater than 90%: RDS, Roche TesTcup, SRT, and Triage. In oral fluid, the evaluation was hampered by the low number of positive samples (Table 5). In addition, the sensitivity of Drugwipe was too low. For sweat, the number of samples that could be evaluated was also small, and the evaluation was done with positive samples only. The accuracy of Drugwipe was 77% (Table 6).

When comparing the GC-MS analysis of opiates in different body fluids with the GC-MS analysis of blood samples, oral fluid had slightly better accuracy than urine (Table 3). Six of the 11 on-site tests met the analytical criteria (Table 4): RDS, Cortez, SRC, SRT, Status DS, and Triage. With oral fluid, the on-site tests showed less accuracy than with urine tests (Table 5). The sensi tivity, in particular, was too low. An ideal oral-fluid test should have a detection limit of 2-5 ng/mL for opiates.

3.4.3. Practical and Operational Aspects

When the necessary facilities were available (e.g., a sanitary van), urine could be obtained relatively easily at the roadside. When the facilities were not available, obtaining a urine sample was a problem, and it could be time-consuming if the driver had to be brought to a suitable facility. In some cases, the volume of urine obtained was low and was insufficient for certain test devices. Some countries clearly stated that sampling urine at the roadside was unacceptable. A clear majority of countries preferred oral fluid as the matrix for roadside testing, while one country favored sweat and one country favored urine. The methods for obtaining oral fluid needed further improvements. Wiping over the tongue seemed to be a well-accepted technique, but in this case the analytical detection technique needed to be very sensitive. Sampling oral fluid with dedicated devices also gave rise to the following problems:

• It was sometimes messy and uncomfortable for the subject;

• The cooperation of the subject was needed (in some cases, intentionally or not, the subject swallowed the collection device);

• Oral fluid was sometimes viscous and could not be used with some devices.

Moreover, dry mouth was a frequently encountered problem in drug users. Sampling was then even more difficult and time-consuming. However, in the present evaluation, obtaining oral fluid for testing was successful in nearly all cases. Overall, sweat and oral-fluid sampling seemed very well accepted by the subjects, much moreso than that of urine or blood.

3.4.4. Discussion

Eleven different on-site test devices for detection of illegal drugs in urine were evaluated. Most of the urine test devices only reached accuracy levels of approx 90% when correlated with the blood results. This could be a result of the discrepancy in temporal distribution of drugs in different body fluids. A much better accuracy rate is reached when the urine results of roadside test devices are correlated with urine results from GC-MS analysis. In this case, several test devices surpass the 95% accuracy rate for some drug classes. However, some limitations of the study design, which is mainly dictated by the different legal situations, must be pointed out:

• The analytical methods used in all the countries were not identical; the evaluation of the devices was done in different places—at the roadside, in the police station, or in the laboratory;

• The devices were evaluated by different persons, which made their comments on the practical and operational aspects of the study difficult to compare;

• Prevalence of drug use and the selection criteria of the subjects differed among the countries, resulting in variability in the preferential use of different specimens for different on-site test devices in different countries.

In several countries, the ROSITA evaluations were the first experience that police officers had had with roadside drug tests and, despite some problems and disappointments, police officers still liked having the tools to detect drivers under the influence of drugs. Users of on-site tests had also shown great creativity in overcoming some of the encountered problems. The oral-fluid devices available at the time of the study all had practical disadvantages, and the analytical evaluation was not satisfactory. But the need for such devices was so great that in one country, police officers preferred to perform an oral-fluid test that was imperfect, rather than no test at all. In other countries, police would rather use urine tests. Police did not have major objections to collecting specimens. A majority of the countries favored oral fluid as a test matrix. Besides the analytical evaluation discussed here, all test devices that were part of the field trials had also been evaluated by the police with respect to handling, ease of sampling, speed, and overall user-friendliness.

In the "needs and requirements" survey, most police forces in Europe expressed preferences for test devices based on oral fluid or sweat because of the ready availability of the specimen. Interestingly, sampling urine during the field-test phase was not a problem, if appropriate facilities (such as a sanitary van) were available. In other cases, drivers had to be taken to a police station or health center, which took time. In one country, when drivers were asked to give a urine sample at the roadside without suitable facilities, the refusal rate was high. Most police forces in Europe are legally not authorized to obtain a urine sample by force. Sampling oral fluid or sweat was much more acceptable to drivers. The possibility of using sweat as a testing specimen is especially of interest to the police forces.

In some cases, the volume of urine collected was not sufficient for the cup-type test devices (e.g., SRC, TesTcup, RDS). This was a problem in 3% of the cases in Germany. In that respect, RDS has the advantage that the urine can be pipetted onto the card.

In some cases, the calculation of the different analytical criteria was hampered by the skewing of the data toward one end of the positive-negative spectrum. For example, many drivers have been tested with ORALscreen, but most of the results turn out to be negative, leading to good accuracy values despite the fact that sensitivity is insufficient. The accuracy of the on-site tests for oral fluid is not satisfactory when one compares it with the reference method in the same specimen: the sensitivity is between 25% (ORALscreen cannabinoids)

and 88% (Drugwipe amphetamines/MDMA). Specificity is in the range of 48% (RapiScan benzodiazepines) to 100% (RapiScan cocaine). Very high specificity values are again a result of no or very low numbers of positive samples. The performance of Drugwipe for amphetamines/MDMA and opiates in sweat seems good, but very few negative samples were analyzed. More studies will be needed to confirm these findings and to allow a proper evaluation of Drugwipe as a sweat test.

Benzodiazepines are present in oral fluids in extremely low concentrations. In a review by Kidwell (8), the limits of the methods used to detect benzodiazepines in oral fluid range from 0.05 to 5 ng/mL, with the majority being less than 0.3 ng/mL. At present, the sensitivity of the on-site test and of some confirmation methods is poor.

The sampling method for Drugwipe (wiping over the tongue) was appreciated everywhere, because of minimal discomfort and low sample-volume requirement. ORALscreen was considered "disgusting" in Germany and Scotland because of the many complications that occurred during sampling, and in nearly all cases, the fingers of the officers and researchers came into contact with oral fluid. This was certainly less acceptable to them than working with urine. Sampling with Cozart Rapiscan was also problematic. The process took a long time and was cumbersome. It was worse if drivers were able to provide only a limited volume of oral fluid, either because of low oral-fluid production or because of refusal to cooperate. The average duration of sampling with the Rapiscan was 4 min, with extremes between 1 and 12 min. Average total time for sampling plus analysis was 20 min (range was 13 to 33 min), which was considered too long for roadside use. One advantage of Cozart Rapiscan was the availability of excess (diluted) samples for performing confirmation tests in the laboratory. In the final analysis, sampling by wiping the tongue is well accepted, although this process requires a very sensitive detection method because of the low volume and the lack of sufficient sample for confirmation analysis. The other methods all have some drawbacks, and more research is needed to develop more efficient sampling methods.

In terms of practical use, none of the testing devices was fully acceptable to the police officers. In Germany, the acceptance of the oral-fluid tests was much less than any of the urine tests. Drugwipe was considered simple in terms of the training needed. The small sample volume and rapid turnaround time were appreciated; however, viewed less favorably were the availability of only single tests, the difficulty of reading of the results, and the need for water in order to perform the test. Use of the electronic reader was considered impractical in Norway, but was considered essential in all other countries. In Italy, oral fluid was considered quite easy to test at the roadside, at least through the use of Drugwipe.

With Avitar ORALscreen, problems were encountered in the transferal of oral fluid from the sampling device to the test. In Scotland, problems were encountered with the transfer of viscous oral fluid, and some samples failed to migrate to the analytical strip as a result of manufacturing faults. Some problems with reading the ORALscreen were reported. Faint lines were produced, especially for cannabinoids, resulting in difficulty in distinguishing presence or absence of drugs. The multiple pieces of equipment and the need to place them on a flat surface made the use of Cozart Rapiscan impractical for a police officer on a motorbike, and restricted its use to police officers driving a van. The equipment proved to be rather complicated to use, and the total time needed to obtain a result (sample collection, sample preparation, run time) was at least 15 min. Often, it took that long just to collect sufficient oral fluid from drivers under the influence of drugs, as they often had very dry mouth as a result of drug use. General comments from police officers were that the sample-preparation procedures (filtration, pipetting, and handling of sample tubes) were rather complicated, such that previous training in the laboratory was necessary.

0 0

Post a comment