Researchers Harness the Power of Quantum User Interfaces

Ben Shneiderman and Ben Bederson

April 1, 2016 

Researchers announced an astonishing breakthrough by creating Quantum User Interfaces (QUIs) which promise to deliver remarkable increases in user capabilities. The startling idea is to replace millions of user actions with a single action that accomplishes a full month or year of work. A simple metaphor that conveys the power of quantum user interfaces is the well-established global search and replace command. Using a traditional Graphical User Interface (GUI), a large database could be updated by changing thousands of occurrences of “coffee” with “tea” manually while a single global search and replace command could make the change much more rapidly. However, even the best modern GUIs have no way to speed up more complex tasks such as making many versions of a document for different audiences. With a QUI, a user could specify several kinds of drinks and entrees simultaneously, and all versions of the document would exist at once and be available for any purpose. With a QUI, the user can do in one step what might take hours to otherwise accomplish. Researchers have come forward with many powerful examples.

QUIs are different than GUIs which are based on discrete operations. While some GUIs can be automated to speed them up or parallelized through collaborative computing approaches, QUIs work by user interfaces which can be in a superpositions of states. That is, some graphical elements can be in multiple states at the same time – so, for example, a button can be both pressed and not pressed simultaneously. This enables vast speedups by going beyond the Model Human Processor and software architectures such as Model-View Controller. However, they do not violate our basic understanding of Human-Computer Interaction and bedrock HCI principles such as Fitts’ law remains intact.

The power of QUIs is measured by counting the number of GUI commands that are replaced by a single “quantum user interface click”, commonly called QUICKS. Many of the projects yield 1,000 QUICKS but some show the possibility of 1,000,000 QUICKS (1 mega-QUICK). Longer-term hopes are to push toward giga-QUICK and peta-QUICK designs.

Certain problems can be solved much more quickly with QUIs than with traditional GUIs. Even those GUIs that use the best currently known approaches such as Direct Manipulation, Gestalt Theory for visual design, and Collaborative Computing approaches can be dramatically sped up. For example, Doctoral student Ambrose Light’s work at the University of Maryland’s Human-Computer Interaction Lab conveys the exponential increases in user abilities that researchers expect QUIs will bring. Ambrose has documented 3 mega-QUICK speed ups in complex tasks such as replacing the work of teams of citizen scientists by executing a single command. By configuring the geographical database to include the possibility of a bird sighting or not of every known species at every location simultaneously, the mechanical data entry of each observation is replaced with simple confirmation. This command is called the “gather” operation since it can bring millions of bird sightings into a nearly error-free database within 1-2 hours.

Similarly, doctoral student Hadassah Agrawala at the University of Washington, has demonstrated a QUI with unheard of power to help job-seekers by listing themselves as both interested and not interested at every company that has open jobs listed which is expected to dramatically reduce national unemployment by a full percentage point. This QUI enables job-seekers to issue a single command that returns a precisely selected set of job descriptions from dozens of independent databases. Then the QUI filters them into a single file, organized geographically, and ranked by similarity to the users’ abilities. Hadassah reports performance in the 50-70 mega-QUICK range. She has launched a startup company, QUICKJOB that has already drawn $11.7M in funding, and will provide a public service, for a fee, within three months.

Even as QUIs become commercially available, other researchers are already racing off to the even more ambitious String User Interfaces (SUIs) that depend on the vibrational structure of queries as they spread through exo-scale databases. SUIs would replace hierarchical file structures with wave-like superposition files, in which even high-dimensional intersections are resolved in less than a millisecond. By using relativistic compensations, searches that used to take hours can be performed by novice users in seconds. While still expected to be 5 years away from commercialization, SUIs can combine action and reaction in a single operation. QUICKJOB has already filed a provisional patent application showing how employers can react by simultaneously offering and not offering jobs to all the applicants and having the vibrational equilibrium perform the proper matching so exactly the right applicants end up with job offers. Skeptics are not yet convinced that these results are guaranteed to perform the right matches, and are currently modeling whether a combination of money saved through speedups and legal protections through provisional employment contracts would provide benefit overall. Clearly this approach is promising enough as there has been a chain reaction of openly published papers (see arXiv.org) claiming ever greater abilities for SUIs.

Worldwide attention is gathering for the July 16-18 Symposium on Quantum User Interfaces: New Technologies (SQUINT) to be held in College Park, MD. Conference Chair Ben Neb hints at further breakthrough announcements that have enticed participation by large numbers of journalists, venture capitalists, and government agency funders.

Seeking nominations for SIGCHI Awards

SIGCHI seeks nominations for its five major annual awards.  All nominations are due by November 15, 2015.

SIGCHI identifies and honors leaders and shapers of the field of human-computer interaction with the SIGCHI Awards. We recognize individuals who have contributed to the advancement of the field of human-computer interaction. There are five kinds of SIGCHI Awards that are selected by two committees based on your nominations, so please submit them! We encourage you to consult the SIGCHI Awards web page (www.sigchi.org/about/awards) for past awardees and more detailed award descriptions.

The SIGCHI Achievement Awards committee selects the following three awards. Submit nomination material to sigchi-achieve-awards@acm.org, to the attention of Steve Feiner (CHI Academy Member) who is the committee chair:

1. SIGCHI Lifetime Research Award: Individuals who have contributed the very best work in shaping the field, SIGCHI's highest honor for research contributions.
2. SIGCHI Lifetime Practice Award: Individuals who have made outstanding contributions to the practice, application, and understanding of human-computer interaction, SIGCHI's highest honor for practice contributions.
3. CHI Academy: Individuals who have made substantial contributions to the field of human-computer interaction should include a summary of the person's contributions with evidence of the cumulative contribution, influence of the work on others, and development of new directions.

The SIGCHI Service Awards committee selects the following two awards. Submit nomination material to sigchi-service-awards@acm.org, to the attention of Ben Bederson (SIGCHI Adjunct Chair for Awards) who is the committee chair:

1. SIGCHI Social Impact Award: Individuals who through their work have made substantial contributions to pressing social needs.
2. SIGCHI Lifetime Service Award: Individuals who have contributed to the growth and success of SIGCHI through extended service to the community over a number of years.

Nominations should include:

• a brief summary (maximum one page, preferably a PDF) of how the nominee meets the criteria for the award.
• (optional) a link to the nominee's CV, if available
• (optional) names and contact information of people who both endorse and are knowledgeable about the qualifications of the nominee.

All nominations must be submitted by November 15, 2015. General questions can be sent to sigchi-ac-awards@acm.org.

Books I've Read Visualization

I am the product of my experiences, and a significant part of my lifetime experiences are the books I have read. Strangely enough, I have kept track of every non-work (and many work-related) books that I have read since 1991. I used to write these down on paper, but a few years ago I started keeping track of them on goodreads.com. Goodreads is a fine service, and offers a nice way to see what your friends are reading.

But Goodreads does not offer any way of seeing any overview of the books one has read, and loses the opportunity to gain any insight into a person's overall readings.

So I created an interactive visualization to try and understand what the 246 books I read over the last 24 years actually were. Go try out the interactive visualization (which is not mobile friendly), and then finish reading this. The visualization is based on Keshif, a free and general tool built by Adil Yalcin, a grad student working with me. It works by showing "facets" of the dataset and supports very lightweight exploration by mousing over the facets to see how they interact with other facets. For example, in 2014, I read a lot by Hugh Howey (I loved Wool!), and those books were mostly written in 2009 or later. Also, I clearly got re-hooked on science fiction.

This approach lets you see things like most commonly read authors and genres. Also when I read books. For example, you can see that I read a lot in 1992 right after I finished my B.S. during my "gap year" when I lived in Alaska. I also started reading significantly more books 3 years ago - exactly when I bought my Kindle - which seems to be consistent with what others have found.

Anyway, take a look and see if you can learn anything else about me. Also, the code is all freely available, and shouldn't be too hard to adapt to your books if you are a bit of a web hacker.

Some technical notes:

• Data comes straight from Goodreads API, but I downloaded it to avoid cross-site permissions issues.
• Goodreads does not provide book genres through their API, so I manually created those in a google docs spreadsheet and load them separately and merge the two data sources.

New Google Drive design forgets about desktop users

Update 8/15/14: Google has fixed this! The list view is now properly dense with about twice as many items as the thumbnail view. Thanks Google!

I was excited to see Google post their new design for Drive. Given that this is a major new design with no doubt significant thought behind it, I was surprised - and dismayed - to see that the two primary views of documents (list view and thumbnail view) show the same number of documents. The textual list view, typically designed for efficient scanning of large numbers of items based on full title and some metadata shows no more items than the image based view.

New Google Drive with list view on left, and thumbnail view of same documents on right.

This may seem like an esoteric design detail, except it isn't. It goes to the heart of design decisions where there is a single design that must serve a huge community (probably hundreds of millions of users). Depending on your view, design is driven down to the least common denominator, or perhaps it is better viewed as regression to the mean.

My guess is that the whole redesign is built to better support touch screen systems given the huge vertical space between items. This is great for all those phone and tablet users, and even the occasional Chromebook Pixel user. But what about us folks in an office with a big screen and a mouse? I know that Apple serves the causal computer using market first, but now even Google has abandoned the regular office worker!

The reason this is an issue is because the trade-off between these two approaches - at least in my view - is primarily between the density of items, and amount of interaction. That is, one view (typically the thumbnail view) should have fewer items with a sparser layout requiring more interaction to see more items. But the painful and expensive scrolling operation (in comparison to moving your eye) is sometimes worth it because of the additional information made available by the additional information per item. The other view (typically the list view) should have more items with a denser layout so you can quickly scan through many items without having to scroll as much.  Arguable, the additional owner and date information in the text view provides the relative advantage, but make no mistake - the text view is a very sparse design.

No doubt, the major vendors are wise to provide an excellent experience for mobile and touch users. But PLEASE don't abandon us desktop users. Don't make the mistake that Microsoft did with Windows 8. You can not design a single experience for all people and all form factors. You must detect what systems people are using and provide an optimized experience for that design. Or at least offer customization options.

Interface designers that build a single experience for all humans on the planet or doomed to disappoint huge segments of the market. Don't let this be you.

My new job: Associate Provost of Learning Initiatives

I am excited to start a new job on campus as Associate Provost of Learning Initiative and Executive Director of the Teaching and Learning Transformation Center.

For those of you wondering why I would give up the world's best job as a professor developing the future of interactive computing technologies: Technology is on the cusp of significantly changing the way that people experience higher education (actually, it already has - from MOOCs to GitHub to YouTube and StackExchange, Twitter, Skype & Google docs...). Now it is poised to change the classroom experience as well. I see this role as an opportunity to apply all that I have learned from my work in the field of HCI over the past 20 years directly to the campus that I love and want to see thrive. From enhanced user experiences to analytics that balances transparency and privacy across multiple stakeholders, this is HCI's time to improve our campus. And if you want to know how I got to this point, then learn about the HCIL - the world's best HCI research group: http://www.cs.umd.edu/hcil/. (Oh, and of course I can always revert to being a regular faculty member if this doesn't work out...)

-----

Dear University of Maryland Faculty:

I am pleased to announce the formal establishment of the Teaching and Learning Transformation Center (TLTC) and the appointment of Professor Benjamin B. Bederson as the new Associate Provost of Learning Initiatives and founding Executive Director of this new center.

The TLTC will be the hub on campus that brings together support, incentives, infrastructure, assessment, and innovation for the University's educational mission.  It will provide leadership around teaching and learning, including the use of technology to transform the students' and instructors' experiences, both in the classroom and in an online environment.  The TLTC will integrate the Center for Teaching Excellence from the Office of Undergraduate Studies and the Learning Technologies and Environments group from Division of IT.  It will also add a campus-wide resource in assessment and learning analytics.  Together, this critical mass will further promote innovation and excellence in education at both the graduate and undergraduate levels.

Dr. Bederson (www.cs.umd.edu/~bederson/) is a Professor of Computer Science and prior director of the Human-Computer Interaction Lab at the Institute for Advanced Computer Studies (UMIACS) and iSchool.  An ACM Distinguished Scientist, his research is on technology-enhanced education, interaction strategies, crowd sourcing, mobile device interfaces, visualization and digital libraries.  He is a widely respected innovator, entrepreneur, and scholar in interactive systems who has worked on a wide variety of interdisciplinary projects, published broadly, and founded a successful mobile apps company.

Please join me in welcoming Professor Bederson to his new position.  Many of you will no doubt hear from him in the coming months as the center is developed.  He also asked me to say that he welcomes hearing from you about your concerns and ideas about teaching and learning on campus.

Mary Ann Rankin
Senior Vice President and Provost

Google Sheets as website backend store

Two summers ago, I spent an inordinate amount of time rebuilding my home page using a database and Django so I had a good UI for entering new data. It was a fun learning project, and it worked well, but was very tricky to get configured on my CS dept. controlled server. Fast forward to now. Some small thing broke and I realized it was going to take a long time to figure well enough how things worked so I would be able to fix it. And even if I fixed it, I'd surely have to go through this again the next time it broke.

Complexity is poor design. So I decided it was time to kill the database and Django and go for a much simpler solution.

I am now using Google Sheets as my backing store, and have straightforward Javascript that queries the store. The end result is that I deleted TONS of code, removed all the configuration junk, and now have a collaborative editing model for the backend data that drives my (simple website).

My home page uses 3 spreadsheets as data stores - News, Projects, and Recent Publications. Each one is implemented similarly, so I'll include the code here for my News in case anyone wants to try this. It relies on Google's visualization library since Google wrote this to make it easy to use their charts and visualizations based on their spreadsheet data.

In your HTML, include Google's Javascript API:

In your Javascript, load Google's visualization library

// Load Google visualization code to support spreadsheet data fetching

if (typeof google !== 'undefined') {

    google.load('visualization', '1', {'packages': []});

}

Make sure your spreadsheet is publicly readable, and that store the spreadsheet URL in your code like this.

var newsSource = 'https://docs.google.com/spreadsheet/ccc?key=0Al_L57Iqw4WSdFhUYkFrMUhsRGFXdUhfUTJydU5JekE';

Then, the following functions shows how I got the data from my spreadsheet, complete with a SQL'ish query mechanism, and displayed it. You'll have to modify it for your site, but the approach is pretty straightforward.

function loadNews(limit) {

var query = new google.visualization.Query(newsSource);

query.setQuery('select D where A=true order by B desc limit ' + limit);

numNewsLoaded = limit;

query.send(newsQueryResponse);

}

function newsQueryResponse(response) {

if (response.isError()) {

alert('Error in query: ' + response.getMessage() + ' ' + response.getDetailedMessage());

return;

}

var dataTable = response.getDataTable();

news_data = convertToArray(dataTable);

newsLoaded(news_data);

}

function newsLoaded(data) {

var str = "

";

for (x in data) {

var item = data[x];

var html = item[NEWS_HTML];

str += "

" + html;

}

str += "

";

if (numNewsLoaded == initNumNews) {

str += "older news...";

}

$("#news").html(str);

}

function convertToArray(dataTable) {

    var numCols = dataTable.getNumberOfColumns();

    var r,c,arr = [];

    arr.length = dataTable.getNumberOfRows(); // pre-allocate for speed

for (r=0; r

var a = [];

        a.length = numCols; // pre-allocate for speed

for (c=0; c

arr[r] = a;

}

return arr;

};

Note that Google's new spreadsheets aren't supported yet. You can read more about Google's support for these kind of data queries.

Thanks to grad student Adil Yalcin for helping me figure out this trick.

Guest Post: Ben Shneiderman on "Her"

My colleague, Prof. Ben Shneiderman wrote the following review of the recent movie "Her". I post it because it reflects Shneiderman's long-time argument that anthropomorphizing computing systems is neither effective nor appreciated by users. Although "Ask Alex" on www.united.com is an example that has been running for at least 1.5 years so why is it apparently successful there? And while Siri may not be in the news much anymore, Apple hasn't pulled it from iOS either. Time will tell if Shneiderman's prediction that all such anthropomorphic technologies will disappear.

Shneiderman's observations that "the need for physicality, exclusivity, and full disclosure are central to human relationships"may seem obvious when you read his argument, but it is this clarity of recognition that makes the review valuable. This much longer review in Wired where these issues never take center stage make the point that the "obvious" is only such if you are tuned to what is important.

---------

Review of Her (the film)
    Ben Shneiderman (1/12/2014)

This creative, clever film depicts current fantasies about how an OS could have a “real” loving relationship with a person.  The emotional empathic tone of Theodore and the charming voice of Samantha make compelling conversation, that seduces Theodore and the audience for a while.  But Samantha’s total misreading of human physicality emerges when she sends a surrogate body, which Theodore rejects.  When she reveals that she is having 641 simultaneous loving discussions, their relationship hits a brick wall.

The need for physicality, exclusivity, and full disclosure are central to human relationships, but Samantha’s programmers failed to realize even those basic requirements.  Of course, they fail in so many other ways, but the screenwriters cleverly manipulate the audience to go along with the perfect partner fantasy.  Scarlett Johansson skillfully says all the right things with delightful empathic intonation, reminding us of our powerful attraction to supportive comments, thoughtful suggestions, vulnerable phrasing, and loving teases.

The filmmakers are right in reminding us of our growing infatuation with mediated relationships.  Real people need real love, not artificial substitutes or technology distractions.  So, will audiences turn off their phones and turn on to their partners, family, and friends to express genuine feelings?  I hope so.

This film continues to promote the misguided belief that humans want technological partners and avatar-based machines.  Consumer rejection of anthropomorphic technologies has been consistent for the past half century (Tillie the Teller, Postal Buddy, Ken the butler, Clippie, Ananova, etc.) but new generations of designers keep coming up with fresh versions, which will also disappear.

This future technology portrayal continues the 50-year old Star Trek/Star Wars tradition of using voice communications to let the audience know what is happening, and use emotional tone effectively.  However, the future of computing will be more visual than verbal.  Voice is important for human relationships, but can’t keep up with the human mind’s desire for information abundance and swift decisions.

The film also might trigger reflections on the merits of autonomy and interdependence.  The mistaken model of autonomous machines has dangers, and the need for human balance between autonomy and interdependence needs modern reinterpretations to fit evolving technology.  Human locus of control is an essential design guideline, even as technology sophistication grows.

Prof. Ben Shneiderman          ben@cs.umd.edu
Dept of Computer Science   301-405-2680  
A.V. Williams Building             www.cs.umd.edu/~ben
University of Maryland          www.cs.umd.edu/hcil
College Park, MD 20742        Twitter: @benbendc

Google I/O 2013 Trip Report

Since a handful of people have asked me about Google I/O, and it isn't a conference that a lot of academics go to, I thought I'd share my experience with this group.

First off - why did I go?  The answer is because I didn't go to CHI so I wasn't worn out, and because Google asked.  They offered me a $300 ticket (compared to the normal $900 which is also very hard to get).  The primary reason I wanted to go was to experience the silicon valley big company event. I was interested of course to learn about Google's view of modern web technology, but mostly I just wanted to feel what this kind of event was like. (And full disclosure, I have research funding from Google.)

So, first let me comment on that feeling.  It was pretty much what I expected - and totally surreal. It was VERY well organized with hundreds of Googlers and staff helping everywhere.  Academics were treated specially (with a short special line, a special floor at an evening party, an invite to Google's SF offices for a tour, etc.). And it was seriously geeky. I was a distinct minority - caucasian and old.  There was very little gray hair and very few women. And a lot of Asians of all kinds. The keynote was a lot like a rock concert. Super crowded - intense spirit. Thousands of people holding up their phones to take pictures of the screens showing fluid blobby animations to rock music before the start. And every one yelling a count down to the start (illustrated on the displays.)  I almost laughed at that part - I guess you can tell my age. I enjoyed it, but just couldn't get *that* worked up about it :) The 3 hour keynote (and all the talks) was very well orchestrated and presented. I really enjoyed hearing Larry Page give his talk. Unlike the other highly scripted talks, his was clearly quite heartfelt. He was doing his best to step out of his demanding role and relate to his techy audience, and I think he succeeded. (Of course, cynics could argue that he was just very good at scripting the feeling of non-scriptedness...)  He largely talked about the importance of being innovative and doing good - and stop the cat fighting that tech companies do. And then without any warning, he took 30 minutes of questions and there was a pretty interesting discussion - probably my favorite part of the whole conference.

There was, of course, excellent free food (breakfast and lunch). Free coffee. Free snack area. And a free Chromebook Pixel (retail $1,450) - meaning they gave away something like $10m worth of them. This event was not a money maker! At least not directly. They of course spent 3 days exciting 6,000 developers about using Google tools to help pursue Google's vision. You may have heard that Google is making their interfaces simpler - removing more stuff, predicting more stuff, and using more of your personal data. Some people will no doubt find that very scary and objectionable. But clearly lots of people (including, I admit, me) find it an acceptable trade-off.  And their demos were astoundingly good. For any of you that remember Microsoft's painfully embarrassing voice recognition demos a few years back - Google ACED them. There was 100% recognition rate of live demos of the speaker talking into an actual phone, speaking fairly complex natural language at regular rates in a very noisy room of 6,000 people.  And if you remember "Put That There" from 1979, then it was all the more impressive when they had good pronoun connectivity back to previous utterances.  Expect to see a lot more voice recognition built into every Google product.  (And no more clicking - soon you'll be able to just say "ok google" and start talking when most google products are open.)

Glass was a big deal (but hardly mentioned in the keynote).  Hundreds of people were wearing them. There was a place to try them out (which I did). And a number of talks on Glass (videos available for many of them.) Personally, I find Glass intriguing - but in its current form I think it is unlikely to be used outside of niche areas and geeks (beyond a startup phase). I predict they will be like bluetooth headsets. They will probably be everywhere next year until people realize that it is too annoying to wear something all the time (and charge them, etc) for the infrequent times when the value add over pulling out your phone is worth it (again, except for the geeks and niche uses). OTOH, if the cost was much less (i.e., it was built invisibly into regular glasses, had inductive charging, etc.) then that value calculus could be very different.

And then there were 2.5 days of talks, which I got somewhat bored of by the second day. Two of the talks I most wanted to see were overfull and I couldn't get in. It was very crowded and difficult to move around between sessions. And many of the others were just too deep about topics I wasn't that interested in (i.e., APIs for every Google product). It would be greatly rich for people building web startups using all of those tools - but academic research tends to be much more selective. And when there is something I actually want to use, it is easy enough to learn it with public resources.

So, bottom line, I am glad I went for the overall experience. But I'm not sure that I would recommend it to most academics (especially since most of the talks are available online.) But it was fun. And Google probably succeeded in their biggest goal which is that I left feeling more positive about Google than when I arrived.