Date: Sat, 2 Jun 2007 18:11:18 -0700 From: To: Subject: EVT '07 Here are the reviews for your submission to EVT 2007. Title: Extending prerendered-interface voting software to support accessibility and other ballot features Authors: Ka-Ping Yee Email: ping@zesty.ca The author describes the design and implementation (in great detail) of the Pvote system, a compact, Python-based voting system using pre-rendered user interfaces as well as a simultaneous audio feature -- a follow-on to the author's previous paper, describing the PRUI concept in EVT '06.  This paper also discusses security, both in terms of complexity minimization and in terms of expert review, and offers some philosophical discussion on how a system might make different trade-offs in this space. Many readers will find the extensive details presented in this paper to be excessive.  Sections 5.1.2 through 5.2, and Figures 1 and 2 are likely to be skipped by many readers.  Still, if you can't publish this in EVT, where can you publish it? It's not unreasonable to describe this paper as a modest improvement over the author's EVT '06 paper, but it's still interesting and valuable.  If anything, I would like to see less of the nitty-gritty design detail on Pvote and instead I would prefer to read more about whether it's a good or bad design decision to include the entire Python interpreter and the software stack that runs below it, along perhaps with a ballpark estimate of the code expansion that would result if Pvote were to be translated to C and run on the "bare metal" (or with minimal support from a suitable embedded application toolkit). ======================================================================= This paper extends the pre-rendered interface research to better address accessibility.   This is very useful, important work and the paper should be accepted to EVT. The fundamental idea of separating out the UI design and rendering is one that should be publicized.  The work takes a step forward in carefully considering universal design and accessibility requirements.  While the paper is good as it stands, I identified some areas that could use further work.  I suggest that some of these be mentioned in the conclusion as part of a discussion of future directions/research.  These include the following.  The paper notes that one use case--one paper ballot was used to motivate the work.  I would suggest that a selection of 5-10 different, additional ballots be looked at: there are collections around such as the Niemi collection on line at vote.nist.gov    I wasn't sure if the audio recordings could be synthesized speech or only human speech: this should be addressed. The paper discusses designing for timeouts--I suggest looking at the new VVSG draft to see if the design is compatible with the timing requirement in the usability/accessibility chapter can be addresses.   Low vision (e.g. font and contrast adjustments) needs to be looked at.  Also, how would you use PRUI for alternative languages (this sounds trivial, as the tool should be language independent)?   Finally, how would an election official or vendor use this tool to develop the ballot.  How hard is it to learn the description language, get the ballot rendered, and iterate to get it right? ======================================================================= Looking at this paper from the perspective of someone concerned about the voting experience, and not just security or technology, it's encouraging to see any attention paid to accessibility, since this legal requirement is so often ignored. The general goal of simplifying an electronic voting interface and separating ballot presentation and marking from ballot counting is a good one. Beyond it's security value, this would also allow novel user interface/interaction techniques to be used on existing machines, since the ballot is separated from the rest of the system. I'm glad to see that the authors have (even if only as an addition to their earlier work) begun to consider the functional and user requirements for a voting interface. It's still not clear whether this system could handle a ballot of even average complexity, but I take this as a sign of work in progress. But... I'm disturbed that there are no signs that the authors have made any attempt to learn about the rich body of work in HCI or user experience (beyond contacting one advocate). Too often, the authors seem to have set their sights low. For example, they say it "should be possible" to define a "reasonably usable" ballot with synchronized audio and video and that a single ballot definition for both visual and audio interfaces "should be possible." This is hardly rocket science, or novel work in user interface design. Why not a more concerted multi-disciplinary approach to a multi-disciplinary problem? [Note to the authors: If you are really interested in universal design, you might want to investigate the work of the Trace Center at the University of Wisconsin, the Center for Universal Design at NCSU, the Human Computer Interaction Lab (HCIL) at UMd, and the papers from the ACM conferences on universal usability.] =======================================================================