Talking maps

The way our brains think about places is very different to the way computers store map locations. At the University of Melbourne, Dr Stephan Winter and his colleagues are investigating ways to make computers better at interpreting our spoken directions. It could make voice assistants more useful and reduce the time it takes emergency services to reach the scene of an accident

When you want to go somewhere and do not know the way, you can ask your phone for directions. But you might need to know the exact name of the place, or find it on a map, before Siri or Google Assistant can help. Dr Stephan Winter and his team at the University of Melbourne are trying to give computers the ability to understand us when we describe places. This work is important because the outcomes of this research could help improve many different aspects of our lives, from refining the way we buy tickets for the underground (metro) to reducing the time it takes the emergency services to reach the scene of a crime or accident.

WHY CAN’T COMPUTERS UNDERSTAND THE WAY WE DESCRIBE PLACES?

We find our way around by storing information about places in our brains. Scientists have found that particular cells in the brain represent particular places. Other cells represent directions and distances. The combination of these things – identifying where you are, where other places are, and how to get from one place to another – is crucial for our survival. The cells form a kind of map in our brain that represents the relationship between one place and another. Your bedroom, for example, is a place. You might describe it as the second room on the right upstairs in another place: your house. Computers, on the other hand, currently do not understand places like this. Instead, they are programmed with coordinates – a set of numbers that pinpoint exactly where a place is on a map.

HOW IS STEPHAN’S RESEARCH BRIDGING THE DIVIDE?

Stephan and his colleagues are working on ways to help computers understand your spoken descriptions of places. To do this, they are developing so-called “knowledge bases” that store not only coordinates but descriptions of a place and other information that we might use to get there such as the names of these places and the relationships between these places. For example, you might describe a cafe as being “opposite a bakery” or “two streets away from the park” or “just behind a school”. The scientists are working out how all this information, stored in a knowledge base, can be used by a computer to give you clear, spoken directions such as “You’ll find the cafe next to the large glass library building”.

WHAT MAKES THIS DIFFICULT?

Let’s say you buy a phone from a high street shop called “Smart Phones”. Do you always call it “Smart Phones” or simply “the phone shop”? If you use a different name to the one stored in a computer’s memory, how will it know which phone shop you are referring to? There may also be a second Smart Phones branch in another town, so the computer must apply reasoning to work out that you are likely to want the one nearest your home. Computers will also need to understand the relationship between places, just like we do. A box office, for example, is inside a cinema. But a cinema may also be inside a shopping centre.

A computer needs to understand all of this so that it can send you to the right place: first to the shopping centre and then to the cinema. Plus, we are often quite vague when we describe where places are. We might say that Watford is “near London”, but that is rather different from saying the supermarket is “near the bank”.

HOW COULD THIS RESEARCH BE USEFUL?

It could save lives by helping the emergency services locate the scenes of accidents more quickly. Location information (GPS) from smartphones is not always available, and so an operator in a call centre must get as much detail as they can from someone on the scene. The caller may also be in distress. A computer listening to the call could help the operator by interpreting the place information provided by the caller. The operator could then pass more precise navigation instructions to an ambulance, saving vital seconds.

The other practical use will be in improving the voice assistants in phones, smart speakers, cars and other devices. Such voice assistants will use artificial intelligence, including knowledge bases. A driverless car will know that when you say, “Take me to my music lesson”, it needs to navigate to your music teacher’s home and check your calendar to work out what time to arrive.

Voice assistants will also provide more user- friendly directions. Imagine, for example, your phone being able to tell you that the T-shirt store is “on the left-hand side of the shopping centre as you go in”. Or a ticket machine in an underground/metro station being able to make sense of your vague description: “I want to go to the museum by the river.” Once computers are able to interpret the way we describe places, the door is open to endless opportunities!

Reference
https://doi.org/10.33424/FUTURUM32

Stephan is investigating ways to make computers better understand our spoken instructions.
© Carl Jackson

Imagine, for example, your phone being able to tell you that the T-shirt store is “on the left-hand side of the shopping centre as you go in”.
© Stephan Winter

Stephan works in the Department of Infrastructure Engineering at the University of Melbourne © University of Melbourne.

When you want to go somewhere and do not know the way, you can ask your phone for directions. But you might need to know the exact name of the place, or find it on a map, before Siri or Google Assistant can help. Dr Stephan Winter and his team at the University of Melbourne are trying to give computers the ability to understand us when we describe places. This work is important because the outcomes of this research could help improve many different aspects of our lives, from refining the way we buy tickets for the underground (metro) to reducing the time it takes the emergency services to reach the scene of a crime or accident.

WHY CAN’T COMPUTERS UNDERSTAND THE WAY WE DESCRIBE PLACES?

We find our way around by storing information about places in our brains. Scientists have found that particular cells in the brain represent particular places. Other cells represent directions and distances. The combination of these things – identifying where you are, where other places are, and how to get from one place to another – is crucial for our survival. The cells form a kind of map in our brain that represents the relationship between one place and another. Your bedroom, for example, is a place. You might describe it as the second room on the right upstairs in another place: your house. Computers, on the other hand, currently do not understand places like this. Instead, they are programmed with coordinates – a set of numbers that pinpoint exactly where a place is on a map.

HOW IS STEPHAN’S RESEARCH BRIDGING THE DIVIDE?

Stephan and his colleagues are working on ways to help computers understand your spoken descriptions of places. To do this, they are developing so-called “knowledge bases” that store not only coordinates but descriptions of a place and other information that we might use to get there such as the names of these places and the relationships between these places. For example, you might describe a cafe as being “opposite a bakery” or “two streets away from the park” or “just behind a school”. The scientists are working out how all this information, stored in a knowledge base, can be used by a computer to give you clear, spoken directions such as “You’ll find the cafe next to the large glass library building”.

WHAT MAKES THIS DIFFICULT?

Let’s say you buy a phone from a high street shop called “Smart Phones”. Do you always call it “Smart Phones” or simply “the phone shop”? If you use a different name to the one stored in a computer’s memory, how will it know which phone shop you are referring to? There may also be a second Smart Phones branch in another town, so the computer must apply reasoning to work out that you are likely to want the one nearest your home. Computers will also need to understand the relationship between places, just like we do. A box office, for example, is inside a cinema. But a cinema may also be inside a shopping centre.

A computer needs to understand all of this so that it can send you to the right place: first to the shopping centre and then to the cinema. Plus, we are often quite vague when we describe where places are. We might say that Watford is “near London”, but that is rather different from saying the supermarket is “near the bank”.

HOW COULD THIS RESEARCH BE USEFUL?

It could save lives by helping the emergency services locate the scenes of accidents more quickly. Location information (GPS) from smartphones is not always available, and so an operator in a call centre must get as much detail as they can from someone on the scene. The caller may also be in distress. A computer listening to the call could help the operator by interpreting the place information provided by the caller. The operator could then pass more precise navigation instructions to an ambulance, saving vital seconds.

The other practical use will be in improving the voice assistants in phones, smart speakers, cars and other devices. Such voice assistants will use artificial intelligence, including knowledge bases. A driverless car will know that when you say, “Take me to my music lesson”, it needs to navigate to your music teacher’s home and check your calendar to work out what time to arrive.

Voice assistants will also provide more user- friendly directions. Imagine, for example, your phone being able to tell you that the T-shirt store is “on the left-hand side of the shopping centre as you go in”. Or a ticket machine in an underground/metro station being able to make sense of your vague description: “I want to go to the museum by the river.” Once computers are able to interpret the way we describe places, the door is open to endless opportunities!

DR STEPHAN WINTER
Geomatics Discipline Leader
Department of Infrastructure Engineering, The University of Melbourne, Australia

FIELD OF RESEARCH: Geomatics and Spatial Information Science

RESEARCH PROJECT: Stephan’s research is investigating how to make it easier for computers to understand how we talk about places.

FUNDER: Australian Research Council

DR STEPHAN WINTER
Geomatics Discipline Leader
Department of Infrastructure Engineering, The University of Melbourne, Australia

FIELD OF RESEARCH: Geomatics and Spatial Information Science

RESEARCH PROJECT: Stephan’s research is investigating how to make it easier for computers to understand how we talk about places.

FUNDER: Australian Research Council

ABOUT GEOMATICS AND SPATIAL INFORMATION

Geomatics involves the collection, analysis and use of geographic information and is known by various names around the world. In years gone by, we would have shared geographic information, such as the layout of city streets, using printed maps. Today, we commonly use digital maps like Google Maps and Bing Maps.

This digital aspect of geomatics is known as spatial information and it is one of the fastest-growing areas of computer science and IT. Computer science is needed for managing and analysing the data that is required to create digital maps. But geomatics involves several other science subjects, as well. For example, mathematics and physics are used to describe the shape of the Earth, the melting of polar caps and rising sea levels. And psychology and cognitive science are required for studying the communication of computer systems with people. If you study for a geomatics degree at university, you are likely to learn about all of these fields.

HOW HAVE GEOMATICS AND SPATIAL INFORMATION SCIENCE EVOLVED OVER THE YEARS?

People have shared maps from the very beginning of recorded history. The tools they developed evolved from cave paintings in the Stone Age to navigational charts in medieval times and photographs in the late 19th century.

Cartographers (map-makers) and land surveyors were the forerunners of today’s geomatics scientists. But the field of geomatics goes far beyond maps. Today, experts are involved in the Internet of Things, which connects sensors in buildings, roads and factories to the internet. Their work spans many different fields of science and involves technologies like robotics, laser scanning, remote sensing and machine learning.

WHY WOULD GEOMATICS BE A GOOD CAREER TO GO INTO?

The industry is in desperate need of graduates with qualifications in geomatics and computing, and that is not likely to change any time soon. Spatial information is essential for many areas of technology and business, from artificial intelligence and software development to engineering, aerospace, property and construction. Geomatics is also essential for research into climate change and for helping to track sustainable development. So, you can see that geomatics gives you a wide range of options when it comes to career choices.

ARE THERE MANY CAREER OPPORTUNITIES?

Yes, but they are spread across a wide range of industries in countries all around the world. There are so many applications of mapping that job opportunities may not, at first glance, appear to have anything to do with geomatics. But they are definitely there!

DO YOU NEED A DEGREE IN GEOMATICS TO GET A JOB IN THIS AREA?

Not necessarily. For people who spend their time developing geomatics solutions or applying specialist knowledge, a geomatics degree is a must. But you might still be working in geomatics even if you do not have a specialist degree. That is because software engineers, surveyors and even graphic designers are needed to apply geomatics in the real world. And then there are other professions that use geomatics: farming, insurance, estate agents, transport, planning, health and many more. You may end up using spatial information even if your job seemingly has nothing to do with geomatics.

OPPORTUNITIES GEOMATICS

• “Are you good with IT? Do you have an interest in geography, science or design – or maybe all three? If you’re answering yes, geospatial science could be the ideal career path for you,” says geospatialscience.com. Check out the website for resources to help you decide whether a career in this field is for you.

• Internships and work experience are another great way to find out whether a career in geomatics, mapping and geographic information systems (GIS), and spatial information is for you. Your college or school should have details of local internship opportunities, or check out studentinternships.com for opportunities in Australia.

• The ESRI Academy has free GIS courses and teaching resources. For example, this free course: “Learn how smart maps are shaping the future” is aimed at secondary school students: https://tinyurl.com/y3fqzujm

• According to Salary Expert, geomatics engineers in Australia earn on average $129,424 a year.

ASK DR STEPHAN WINTER

WHAT DID YOU WANT TO BE WHEN YOU WERE YOUNGER?

I was inspired by exciting stories of navigators and explorers, so I wanted to be a cartographer (a person who draws or produces maps). I thought it would help me never to get lost! But I had a competing desire to become an architect because I loved building with LEGO.

WAS THERE ANYONE IN PARTICULAR WHO INSPIRED YOU?

James Cook. Who could not be inspired by his feat of circumnavigating the world? He crossed vast oceans by observing stars and measuring time. Perhaps even more impressive is the story of Martin Waldseemüller. He dared to put a new continent, America, on his world map without ever having travelled. How could he be sure it was not the 15th century equivalent of fake news? His professional reputation was at stake.

There were also the Cassinis – four generations of astronomers and surveyors. Between them, they produced the first topographic map coverage of France and determined the size of Earth by measuring a meridian arc. And, finally, there’s Carl Friedrich Gauss (1777-1855), whom Wikipedia calls “the greatest mathematician since antiquity”. His work is still the foundation of the discipline, but it was only when I was older that I fully appreciated his achievements.

HOW DID YOU END UP WORKING IN GEOMATICS AND SPATIAL INFORMATION SCIENCE?

It happened by coincidence. The university in my home town offered no architecture degree, so I chose the geomatics degree instead.

HOW DO YOU THINK ADVANCES IN SPATIAL TECHNOLOGIES WILL SHAPE THE FUTURE?

The next big transformation will be automation of transport and infrastructure. Spatial information will play a major role in this. Self-driving vehicles and “digital twins” are just two examples.

Self-driving vehicles rely on highly accurate map data containing unprecedented levels of detail, right down to lamp posts and planned lane closures, that automatically updates in real time. The vehicles also need sensors so that they can position themselves accurately on the map at all times. The sensors must be able to capture every other vehicle, bike, pedestrian and dog around the vehicle. At the same time, these vehicles need to communicate with people – with passengers so that the vehicles know where to go, and with pedestrians to determine who has right of way.

Spatial technologies will also be needed for “digital twins”. These are accurate digital replicas of buildings or infrastructure that are connected to their real-world counterparts with sensors. These connections check the “pulse” of the environment and the people in it. By doing so, they can manage resources, maintenance and safety more efficiently.

SOME PEOPLE FEAR ARTIFICIAL INTELLIGENCE AND IMAGINE THAT THE FUTURE MIGHT BE SIMILAR TO HOW IT IS SOMETIMES PORTRAYED IN SCIENCE FICTION FILMS. WHAT WOULD YOU SAY TO THOSE PEOPLE?

Fear is an emotional response to perceived threats. It’s important for human survival when it comes to your immediate response to something. You can say “no” to something as long as you do not understand the consequences. But I think it is important to get beyond that stage and learn about artificial intelligence because it might challenge your initial responses. Artificial intelligence holds as much promise as it does risk. And, as with any new area of scientific research, it calls for ethical use.

STEPHAN’S TOP TIP

There are many different aspects to the research in this field that require many talents, from communicating with data to measuring, mapping and software development. If any of those interest you, check out geomatics degrees at universities – and keep in mind that it can be called by different names at different universities.