A brief reading of Colorado Secretary of State Election Rule 45 regarding Colorado's Voting System Certification Standards reveals several significant problems that should be resolved. The comments below describe some of these problems and proposes solutions in the form of an improved standards drafting process.
Several different versions of Rule 45 exist on the Secretary of State's web site. The following comments refer to what is apparently the most recent version of Rule 45, labeled "RULE 45 - 10-03-05" [1]. Furthermore, the comments below should not be considered a comprehensive review of Rule 45.
1. The State's voting system certification standards must be precisely written, and many sections of Rule 45 are not. This imprecision make it impossible to convert those into verifiable, testable metrics.
1.1. One example of this imprecision is section 45.5.2.1.1, which states that "the voting system shall exhibit an evolution towards new technologies," an abstract requirement which is never concretely elaborated. Another is section 45.5.2.3.8, which states that "the environment in which all databases in the subsystem are maintained shall include all necessary provisions for security and access control," without describing the actual security and access control provisions required. The practical requirements of these and other sections are unclear to the point of being meaningless. While statements of general guiding principles are important, they must be concretely defined with specific, testable criteria in standards documents.
1.2. Rule 45's performance standards also are very imprecisely defined. They make references to operational terms which have no publicly-accepted meaning, and which are not defined in the Rule's glossary, and extend the meanings of other terms, apparently to include other processes. For example, section 45.5.2.2.1 refers to "counting ballots", but does not define whether this process includes the process of casting ballots, as implied by the inclusion of the DRE requirement in subsection (b), or whether it includes the process of ballot scanning, as implied by the optical scan subsections (a) and (c), or whether it includes the process of vote tabulation, which on many systems is separate from the vote casting and ballot scanning processes. Section 45.5.2.2.2 refers to terms like "election media download" and "ballot style assignment" which are not defined - it's not clear whether these terms refer to pre-election preparations, or to the actual ballot casting process. It is also unclear whether these performance requirements refer to individual DRE or vote scanning machines, as implied by section 45.5.2.2.1; or whether these metrics refer to the voting system as a whole, including all of the DRE or vote scanning machines. Testing conformance to the performance requirements in Rule 45 is practically impossible unless the processes under test are rigorously defined.
1.3. The above examples represent only a portion of unclear requirements in Rule 45. These types of imprecision in a certification standards document is undesirable. It creates loopholes by which vendors may certify equipment to the letter of the Rule that does not meet the intention of the Rule. It also fosters confusion as to the Rule's true requirements.
2. The State's voting system certification standards should define performance requirements that specify the maximum amount of time required to perform real-world tasks required in an election, rather than requiring metrics which are not useful for Colorado elections, as are defined in Rule 45. For example, section 45.5.2.2.1 requires a minimum ballot counting rate requirement of 100 ballots per hour for central count optical scan ballots. Such a minimum requirement may be useful for Hinsdale County, Colorado, with only a few hundred voters; but is not useful for Denver County, Colorado, with hundreds of thousands of potential voters. Vendors should instead be required to demonstrate that their system is capable of completing the specified election action in a reasonable election time frame. For example, a more meaningful version of section 45.5.2.2.1 would require voting systems to complete the vote count in a specified maximum number of hours.
3. The parts of the State's voting system certification standards that pertain to computer security and cryptography should be written and reviewed by experts in those fields. This is not the case for at least the information security sections of Rule 45, which were clearly not written or reviewed by experts in the field. For example, section 45.5.2.7.2 refers to "a minimum encryption requirement of 40-bit encryption." Presumably Rule 45's authors are trying to ensure the privacy and integrity of election records that are transmitted over public networks. But section 45.5.2.7.2 does not do this - in fact, it is technically meaningless. For it to be meaningful, the vendor must provide full cryptographic protocol and implementation details, including details on key generation, entropy collection, message authentication, and sender and recipient authentication. These details must be reviewed by experts in cryptography and information security.
4. The State's voting system certification standards should require mandatory compliance with the EAC's Voluntary Voting System Standards [2] - Rule 45 does not require this. These standards are intended to represent the state of the art in voting system standards, and unlike Rule 45, have received nationwide public review. If Colorado's standards are to live up to the Secretary of State's description of them as "one of the most challenging and thorough programs in the country," they should at least require compliance with the latest standards guidance developed by the Federal Government.
5. The State's voting system certification standards should explicitly forbid vendors from submitting "confidential" or "trade secret" information for certification as documentation or application responses to ensure maximum openness and transparency, and Rule 45 does not do this. A vendor could conceivably claim "trade secret" status for significant portions of the documentation required to comply to the rule. This would prevent the public from engaging in any meaningful review of the voting systems, and would conflict with the Secretary of State's commitment to an "open and transparent" voting system certification process [3]. The Colorado voting system standards should forbid the vendor from claiming "trade secret" status for any documentation provided to the State examiners, given the special public trust requirements for voting systems.
6. The State's voting system certification standards should include a glossary which includes all of the terms in the standards which do not have clear public usages, and should build on glossaries already created in the federal voting system standards. But as noted previously, Rule 45's glossary does not contain entries for terms like "election media download," which do not have commonly-accepted unambiguous meanings in elections. Additionally, terms like "Ballot Image" are ambiguously defined. For example, it is unclear whether a "ballot image" refers to cast vote records, or to graphics files depicting the scanned paper ballots from optical scan systems.
7. The State's standards should require vendor systems to support live auditing procedures [4], which Rule 45 does not require. Live auditing is the process of continuously evaluating the accuracy and functionality of election systems during an election using live ballots. Without mandating vendor support for live audit techniques, the accuracy of the election system can only be tested before or after an election, and such testing is inadequate to assure election accuracy.
Many of the principles that motivate Colorado's voting system standards seem well-intentioned. However, the specific manifestation of these standards in the present Rule 45 does not measure up to these principles. It is my belief from a cursory examination of Rule 45 that the problems cited above are not simply endemic to the sections quoted above, but exist throughout the document. This suggests that the public would best be served by committing to a different standards-making process than was used for the Rule under discussion.
A better standards-making process would include the following principles:
- It should be continuously edited by staff experienced with the precise language necessary to define standards.
- It should involve independent subject matter experts outside the Secretary of State's area of expertise.
- All drafts should be released to the public for ongoing comment throughout the drafting process.
- The document should strive to avoid duplicating existing Federal standards, and should draw on existing work, both at the Federal level and standards work by other states. In particular, the State should mandate compliance with the existing EAC Voluntary Voting System Guidelines.
- It should first be articulated in a general set of requirements for voting system performance that are then translated into specific, testable, concrete requirements. For example, a general requirement that "election voting system records and data must be secure from tampering and unauthorized interception" can be translated into specific technical requirements that serve that general principle. Similar general principles and concrete requirements should be articulated for accessibility, reliability, live auditability, speed, and other desirable aspects for Colorado's voting systems.
It is also troubling that Rule 45 was adopted in emergency, before any opportunity for comment or review by independent subject matter experts or the public. Rule 45 seems rushed, and it is a matter of some concern that voting systems may be certified under the emergency rule that would not be certifiable under a more deliberate Rule, either revised from the current rule or rewritten. Therefore, I encourage the Colorado Secretary of State to:
- Revoke the emergency adoption of Rule 45,
- Commit to an open public comment and revision process for a revised voting system standard before it is adopted, and for all future revisions, and
- When revised standards are complete, require immediate re-certification of all equipment certified under the emergency adoption of Rule 45.
Sincerely,
Paul Walmsley paul@pwsan.com
Boulder, Colo.
Footnotes
1. http://www.elections.colorado.gov/DesktopModules/Downloads/download.aspx?tid=501&_iid=193
2. http://guidelines.kennesaw.edu/vvsg/guide_toc.asp
3. http://www.elections.colorado.gov/DDefault.aspx?tid=501
4. http://www.booyaka.com/~paul/ea/eac-20050930/interpretation-live-audit.txt
http://www.booyaka.com/~paul/ea/eac-20050930/tabulation-live-audit.txt
http://www.booyaka.com/~paul/ea/eac-20050930/live-audit-overview.txt
