Thomas Pederson, Dept. of Computing Science, Umeå university, Sweden. E-mail: top@cs.umu.se.
|
Magic Touch in the press
|
As from June 2002, Magic Touch development is temporarily put on ice. Thomas Pederson is currently writing his PhD thesis and will continue with the development of Magic Touch in 2003.
All proposed master thesis projects listed above can become 10- or 20-point projects depending on students' interest. They can also be performed by single students or by two students together, all according to student's preferences. The thesises are to be written in Swedish or English.
| In office environments of today, there is a gap between the physical environment (consisting of books, papers, desks and drawers) and the virtual environment (e.g. the Microsoft Windows environment). This gap can distract users since they often have to manually transfer information and artefacts (paper documents etc.) between the environments which is boring and time consuming. Within the Magic Touch project we try bridge this gap by developing artefacts that exist in both the physical and virtual world at the same time (fig1). We believe that these "Physical-Virtual Artefacts" (PVAs) can help users to concentrate more on creative and fun activities rather than boring routine tasks. |  fig1 |
| In order to create these PVAs we
have designed a system that links physical and virtual
artefacts together (see fig2 below). We currently use a
wireless motion tracker combined with a "home made"
wearable tag reader for tracking how the user organise
things in the physical office space. fig3 to the right
shows a schematic sketch while fig4 shows an early
prototype.
fig2 |
 fig3Whenever the user grabs a tagged object, it is identified by the tag reader. When the user drops the object, the new position of the object is known since the system always knows where the hand is.
|
We believe that by integrating the physical and the virtual world, we make life easier for knowledge workers that today have to create a coherent model of the two separate worlds in their minds, which involves significant cognitive effort. We also believe that this integration can lead to new interaction methods and new tools.
For more information about the Magic Touch system and the research background, don't hesitate to contact Thomas Pederson.
Within the tracked environment (an
office or for instance outdoors), users organise and move around
physical things. The Magic Touch system keeps track of this
organisation and gives the user the opportunity to rearrange and
view his/her objects from different perspectives when in front of
th computer screen. However, not all interaction with the Magic
Touch system can be performed in front of a desktop computer
since the object that should be affected by a particular Magic
Touch operation might be far away from the screen (for instance a
paper document on the top book shelf. In these cases, it has to
be possible for users to interact with the Magic Touch system
using speech and gestures.
The purpose of this project is (a) do define what parts of the
existing graphical user interface within the Magic Touch system
that should be additionally controlled by multimodal interaction
such as speech and gestures, (b) to implement some of the
suggested multimodal interaction commands using available speech
and gesture recognition software.
Theoretical focus: Human-Computer Interaction, post-WIMP user interfaces, Ubiquitous Computing, Augmented Reality, Wearable Computing.
Recommended background knowledge / interest: Human-Computer Interaction, high-level programming languages like C++.
Design and implementation of an automatic document classification system based on user modeling. Based on how users organise their documents in their office (in piles, folders, drawers etc.) the system should be able to make a rough categorization of the documents. The classification should be done based on document attributes like what project the document belongs to, its content, importance in long and short term etc. An important part of the project is to evaluate different ways of categorizing documents. The text content of a document is retrieved not by analysing physical documents themselves, but by analysing the electronic document that the user has linked to the particular physical document (this simplifies the task considerably).
By (a) analysing the contents of documents semantically and statistically, (b) keeping track of where the user chooses to place documents in the physical space (in folders, piles on the desktop etc.), the system should be able to create and maintain a simple model of how documents are related to eachother. Additionally, if time permits, tools for viewing and modifying this model of the "inividual information landscape".
Theoretical focus: Artificial Intelligence: self-organizing maps, neural networks, semantic networks; mathematical statistics, categorisation, text analysis, Human-Computer interaction.
Recommended background knowledge / interest: Artificial Intelligence, mathematical statistics, high-level programming languages like C++.
The goal of this project is to extend an existing VR-visualization of a physical office space and artefacts within this space. An early prototype has been developed using Visual C++ and World ToolKit. The developed system should visualise static things (desks, bookshelves etc.) as well as more dynamic things (paper documents, folders, post-IT notes, pens, users) in a 3D model. The office environment is constantly changing based on where the user move the things. These changes are observed by the Magic Touch core system which also updates a database. The goal of this project is to do an as flexible and realistic visualisation of the office environment as possible, based on the existing database. Furtheron, certain interaction within the VR-scene should be supported like for instance selection and deselection of objects. Programming environment: Visual C++, World Toolkit, Visual Basic. The sensing of physical actions is handled by the core Magic Touch system already developed.
Theoretical focus: Virtual Reality, Information Visualisation, Human-Computer Interaction.
Recommended background knowledge / interest: Virtual Reality programming, high-level programming languages like C++, Human-Computer Interaction.
The goal of this project is to design and implement an Internet-accessible database for distributed Physical-Virtual Artefacts. An early, non-networked, prototype has been developed using Microsoft Access and Visual Basic. The Physical-Virtual database contains (a) information about physical things in for instance a particular office environment and (b) virtual objects (applications, data files), and (c) links between these two kinds of objects. By putting this database on the web, users of computers connected to the Internet will be able to monitor status changes of physical and virtual objects no matter where in the world they might be situated. For instance, an action consisting of putting the outdoor shoes in the closet in your office in Umeå might ring a bell in your colleagues office in Tokyo, telling him/her that you are present. The sensing of physical actions is handled by the core Magic Touch system already developed.
Theoretical focus: Distributed Systems, data communication, Internet issues like privacy and security, object oriented databases.
Recommended background knowledge / interest: Distributed systems, client-server architecture, high-level programming languages like C++ or Java, object-oriented databases.
The Magic Touch core system keeps track of all physical objects (for instance a paper document) and their position in the physical space. It also keeps track of the corresponding virtual object (for instance a web page). The goal of this project is to design a "search engine" that, provided with this present information, allows the user to search for both physical and virtual objects. When searching for physical objects, the search engine should indicate where the found object is located in the physical space by making a servo-controlled laser pointer point to the object. The pointer will be mounted in the ceiling. If project time is not enough, this pointer is simulated in software instead.
Theoretical focus: PC-controlled electronic sensors and effectors, Human-Computer Interaction
Recommended background knowledge / interest: Low and high-level programming languages, Microsoft Windows driver programming, basic electronics.
| The Magic Touch system is currently
using a local area motion tracker for tracking position
changes of objects in the physical space. By using a wide
area positioning system like GPS, and GSM SMS or wireless
LANs, Magic Touch could be used to track objects globally
rather than only in a single room. The goal of this
project is to investigate the possibilities of combining
the existing local area tracking system with other more
wide-area tracking systems like GPS, as illustrated in
fig5. The project involves both theoretical design of a model for combining tracking systems of different granularity, as well as practical design of a prototype system that shows limits and possibilities when using GPS and wireless data communication together with the Magic Touch system. Depending on the interest of the student(s), this project can become oriented towards (a) electronics integration, (b) software development, (c) industrial design, or (d) pure theoretical work. MT06a: Electronics integration into a hand-wearable unitThe goal of this project is to design and develop a wearable system consisting of a simple microprocessor connected to an RF/ID reader (see above), GPS reciever, Bluetooth chip and/or GSM cellular phone and/or wireless LAN. Fig6 shows parts of an initial sketch. This system, worn by the user, will communicate with the internet-based PVA-DBMS server developed in MT04. The design is based on the existing wearable unit (see fig4 above). Theoretical focus: Microprocessor programming, sensor technologies, wireless data communication. Recommended background knowledge / interest: Basic knowledge in digital and analog electronics. Specifically low- and high-level programming of microprocessors, interest in new digital sensor technologies (photo diods, thermal, accellerometers etc.), interest in GPS. MT06b: Development of a GUI for a Wearable ComputerThe goal of this project is to develop a Magic Touch user interface to be run on a wearable computer (see the picture of a MIT Media Lab prototype in fig7). Basically, this project is the same as MT08 but with a specific focus on small-screen interaction and other implications from running the Magic Touch client on a wearable computer in contrast to a desktop computer. For the purpose of the project, an off-the-shelf wearable PC will be bought and used as a test platform. Most probably, the OS running the client will be a Linux-version. Theoretical focus: Wearable Computing, Context Awareness, post-WIMP user interfaces Recommended background knowledge / interest: Good UNIX/Linux knowledge, Basic knowledge in high-level object-oriented programming in C++ or Java, basic knowledge in data communication. Interest in Human-Computer Interaction, Artificial Intelligence. MT06c: Industrial design of the wearable unitThe goal of this project is to open-mindedly consider functional, ergonomical and technical issues with the wearable Magic Touch prototype that exists (see fig4 above) and the emerging versions developed in MT06a and MT06b. Theoretical focus: Industrial Design, Ergonomy, Man-Machine Interaction, Aesthetics. Recommended background knowledge / interest: Good knowledge in industrial design, Human-Computer interaction, Cognitive Science. Basic knowledge in electronics and high-level programming. |
|
To make computers aware of physical alterations on objects that user do, like for instance when the user makes some hand-written notes on a paper document, is a hard task. However, it would largely improve the synchronization of the physical and virtual environment if we would be able to track what changes the user is doing because then we could let the system do the same changes to the virtual corresponding artefact. Keeping the two environments synchronized. One way of doing this is to encourage the use of technology such as the Anoto pattern or IBM CrossPad when performing the changes.
The goal of this project is to investigate the best possible way of incorporating an Anoto pen, IBM CrossPad (or similar) tool into the existing Magic Touch Environment. Necessary hardware is purchased and integrated into the Magic Touch system.
Theoretical focus: Post-WIMP input devices, Human-Computer Interaction.
Recommended background knowledge / interest: High-level programming languages such as C++, Java, Microsoft Windows programming, Human-Computer Interaction.
| The goal of this project is to
develop well-structured object-oriented client program (communicating
with the system developed in MT04) with a flexible user
interface, customisable to many different interaction
devices (desktop PC, wearable computer with display,
without display etc.). The goal of this project is to redesign and extend the current version (see fig8) of the Magic Touch client (a) in order to utilise the new communication architecture defined in MT04, (b) to create a more modular object-oriented and stable platform for future development of the MagicTouch system. Theoretical focus: Computer-Supported Cooperative Work (CSCW), Human-Computer Interaction. Recommended background knowledge / interest: High-level object-oriented programming languages such as C++, Java, Microsoft Windows programming, data communication, Human-Computer Interaction. |
|
[Temporary description]
The goal of this project is to define a framework for modelling what different places in for instance an office stand for, how and when these semantics are connected to the places and also investigate the difference between mobile and fix places. Taking a physical office as an example, different parts of bookshelves might be connected to different projects, a certain area of the desktop might be a very temporary area where objects only stay a short time while other parts of the desk is used for more long-term storage. The area where the outgoing mail is placed is connected to the semantics of sending things away etc. etc. Some places of the office area might be used for reminding. An example of a mobile place connected to semantics might be bag where the office worker puts the things to be used somewhere else. In other words, the physical place which constitutes the volume within the bag is connected to the semantic meaning "whatever is put here, it should be used somewhere else".
However, the analysis of the meaning spaces should be performed on a more general basis and the framework developed should be applicable to more areas than just offices. The theoretical focus of this project makes it involve research-like activities.
Theoretical focus: Architecture, Location-based information interaction, Physical-Virtual Artefacts, post-WIMP user interfaces.
Recommended background knowledge / interest: Good knowledge in Cognitive Science, Human-Computer Interaction. Interest in semantics.
[Temporary description]
The goal of this project is to make the Magic Touch Object Server developed in MT04 (see above) to talk with the ABB Industrial IT server. Since the former speaks CORBA and the latter speaks Microsoft DCOM, a major challenge will be to create a middle layer that can act as a translator between these two communication standards.
Theoretical focus: Data communication, object-oriented design.
Recommended background knowledge / interest: Object-oriented development.
The goal of this project is to design a software system able to track activities that a user performs in an MS Windows environmentcertain menu . Activities of interest are, for example, when the user starts an application or initiates a command, or modifies content or location of files.
Project background: The work done in this project will contribute to two ongoing research projectsand Magic UMEA at the Dept. of Informatics : Touch at the Dept. of Computing Science respectively.
UMEA (stands for User-Monitoring
Environments for Actions) is an approach
based on converting project-specific interaction histories into
project
contexts. It is proposed as a way to minimize overhead, integrate
self-management with communication and management of tools and
materials,
and capitalize upon actual work practices of users.
Magic Touch - until now, focusing mainly on physical activity tracking, the proposed project would expand the system with information about activities in the virtual world as well. For more information on Magic Touch, please see http://www.cs.umu.se/~top/Magic_Touch
Theoretical focus: Tracking of user activities in virtual environments, MS Windows event handling, Human-Computer Interaction.
Recommended background
knowledge/interest: MS Windows OS experience, object-oriented
design.
Working method and payment:
The project can be performed as a performed by single can also be
10- or 20-point project depending on students' interest. It
students or by two students together, all according to student's
preferences. The thesis is to be written in English.
The project can also be performed as a stand-alone software
development project without the commitment to produce an academic
thesis.
In any way, the student(s)
performing the project will receive a certain economical
compensation.
More Information: Viktor Kaptelinin, Dept. of Informatics, vklinin@informatik.umu.se.
Thomas Pederson, Dept. of Computing Science, top@cs.umu.se.
If you are interested in something particular that is related to the Magic Touch project, or if you have an idea that is related to physical-virtual research in general, don't hesitate to contact Thomas Pederson.
fig4
fig5
fig6
fig7 (from MIT Media Lab)
fig8: Magic Touch client v1.0