Rise of the Google machines: the robotics companies involved
By Chris Chesher, University of Sydney
Google recently acquired eight high profile start-up robotics companies, providing strong evidence of a strategy to create breakthrough applications for robotics over the next decade. This strategy is most likely to concentrate on manufacturing and logistics.
Bringing these companies together, Google will need to find synergies between diverse organisations and personalities. This mission will be headed by Andy Rubin, who previously managed the successful Android operating system for mobile devices.
Rubin describes Google’s highly ambitious goal of finding technically and economically viable applications for robotics as a “moon shot”: a highly concentrated effort of an integrated team to create landmark achievements in a field. The mission to put a man on the moon is one clear precedent.
There are many other possible analogies for Google’s robot “moon shot”. Journalist Tom Green, writing in Robotics Business Review, compares Google’s contribution to the robotics industry to the US Defense Advanced Research Projects Agency’s (DARPA) pivotal role in establishing the founding technologies of the internet.
Google’s project might also be compared with Atari research lab, formed in the 1970s to generate innovations in computer game and entertainment technologies. (Unfortunately this did not prevent the massive failure of the company in the mid-1980s.)
An even less appealing analogy is the Manhattan Project that created the atomic bomb in the 1940s. Considering the role of the US military in funding and fostering robotics research, the parallel is not so far off.
Xerox PARC is another corporate that has been highly successful in innovating in the domain of office technologies, but is known most for its failure to transfer research prototypes to viable products.
In expanding Google’s investments in robotics, Rubin will face the challenge of integrating the companies that form Google’s moon shot at Palo Alto, California. What is notable about many of these companies is they are either interdisciplinary in orientation, or highly specialised.
Many of the companies began as spin-offs from university robotics research. The companies that had a spin-off culture will need to transition into being part of a large organisation, with the politics that this entails.
So who has Google bought and what do they do?
Bot & Dolly
This film included sequences that began as computer-generated imagery, which was matched with live action sequences using robotic cameras. In the clip below, robot cameras captured the astronaut’s faces as they spun around in zero gravity.
These images were mapped into the computer-generated sequence. Experimenting at the intersections of cinema, robotics and stage magic, Bot & Dolly produced a stunning performance piece called Box.
Box uses two robots to manipulate screens onto which high definition projectors present geometrical and op-art inspired patterns. A human performer interacts with the screen images, creating a seamless hybrid of multiple disciplines.
Bot & Dolly’s design studio arm Autofuss emphasises its collaborative approach “colliding visual artists with programmers, engineers with designers, storytellers with illustrators, architects with machinists”.
It has produced promotional videos for Google, Microsoft and Adobe. These promotions make heavy use of robotic cameras, motion design, animation and live action production.
Meka is another university spin-off company, coming out of the Massachusetts Institute of Technology Computer Science and Artificial Intelligence Laboratory in 2006. One of their aims is to create highly agile robots that can run quickly over uneven ground.
Holomni is a design firm that specialises in highly controllable caster wheels that can position robots with 360 degree precision. Such a specialised company is likely to produce devices that slot well into any robot that needs precise mobility.
“new generation arm” for robots […] that does for robotics what the Apple II did for computers: get the hardware out of factories and into homes.
Like Holomni, the strategy is the concentrate on one particular component that can be used in a variety of robot applications. Whether Google will pursue this goal of providing wheels and arms to the wider industry, or not, it not yet clear.
A spin-off from high profile robotics company Willow Garage, Industrial Perception Inc produces 3D visual perception systems for applications such as unloading trucks and feeding parts.
They aimed to produce product-level robots that could work at a level and speed comparable to humans unloading trucks (see Casey Nobile’s article in Robotics Business Review). Industrial Perception’s goals seem in line with Google’s goals with their move into robotics.
Their projects have been funded by the US Defense Advanced Research Projects Agency (DARPA).
Boston Dynamics was founded in 1992 by Marc Raibert, a former professor at the Massachusetts Institute of Technology. It was the eighth and last of the companies to join Google so far.
The goal of the competition was to complete tasks to a rescue robot that could drive a vehicle, walk on uneven ground, walk up an industrial ladder, clear debris, open a door, cut through a wall, open a valve and use a hose. The only non-US competitor, Schaft’s robot scored 27 out of 32 points and beat the Boston Dynamics team by some margin.
The Googlefication of robotics research is likely to represent something of a cultural shift for the organisations and employees involved. However, there are common stories for many of the companies. The grounding of much of the research in universities is one clear shared experience.
Each of the companies above has highly specialised applications and well-formed visions. Google wisely selected companies on the basis of some firm instrumental orientation and corporate vision.
In spite of the growing investments in robotics, longer term questions about the future models for robotics in everyday life remain open. How key components — from machine vision to directional wheels, from automated cameras to humanoid rescue robots — might combine into transformative applications is yet to be seen.
Also yet to be known is the impact of Google’s taking cream from the top of a still-young robotics industry.
Chris Chesher writes a blog on the cultural aspects of robotics: Following Robots.
Chris Chesher does not work for, consult to, own shares in or receive funding from any company or organisation that would benefit from this article, and has no relevant affiliations.
An event such as Robotronica, in Brisbane on the 18th of August 2013, reveals how popular robots are with the general public. QUT’s new Science and Engineering Centre was packed with curious families, shuffling from exhibition to demonstration.
Ars Electronica presented an overview of development of the Linz, Austria new media arts centre. They previewed the ‘Spaxels’ performance that would occur later in the evening. Spaxels are moving pixels in space, formed from thirty quadcopter drones with script-controlled lights.
I chose to go to some demos including Paro, a well-known seal-shaped robot used with dementia patients. The way it moves and engages with those around it do seem to make it more than a cuddly toy. Currently in its eighth generation, Paro can sense touch, light, sound, temperature and posture. Each Paro has its own name.
Cheap and cheerful
Another demo was a new robot called the Weaver, a simple three-wheeled, wifi connected vehicle that aspires to be a very economical introduction to robotic principles. The hardware includes a sensors for distance, light, temperature, rotation, microphone and speaker and compass. It features an 8×8 LED matrix and a lamp. Its wheels can turn in any direction. The developers are planning to put the project into Kickstarter within months.
Nao and then
The Nao robot was also there, although the one that I was watching was not working. This happened to me once before, when I first saw the Nao in Korea that had a broken leg. Nao seems somewhat fragile. I understand that faulty robots need to be returned to France for repair. Still, the Nao is an impressive little humanoid.
I went to the talk by Hiroshi Ishiguro. He was in Japan, but his dopelganger, the Geminoid android, was able to give body to his voice. While there was no sense in which the Gemonoid was indistinguishable from the human speaker, it did serve as a focus of attention. There was a kind of fetish value to the android that attracted people at the end of the talk (including me).
In question time it became clear he had a sense of humour. When asked about how he travelled, he said that his head was in the hand luggage, and his body and legs travelled as checked luggage.
It is apparent that Ishiguro has aspirations for his research to become pragmatically valuable, and not simply art practice.
The Diamandini Age
In Old Government House, David and David were trying to get the porcelain-like robot
Diamandini to function properly again. While some visitors were frustrated by her lack of responsiveness, she served quite well as a sculpture until she was back in action. When I returned, she was working — gliding around the parquet floor, underneath the elegant chandelier. She sometimes seemed to be attracted to visitors, while at other times she was repelled.
Finally, I waited for the closing performance by the Spaxels, hosted by Geminoid HI-4, with music by 8-bit hero. The formation of quadcopters appeared in the sky, like some miniature choppers from Apocalypse Now. It was an experimental demonstration of possibilities for multiple quadcopters. Apparently this performance was the first in which the computers running the scripts were on-board the vehicles, rather than remotely operated.
Robotronica was a great success, with many visitors, and a wide array of robots on display.
The search for robots does not always end with finding discrete autonomous actors. The picture is more complex. In March, I travelled 7 hours West to visit the Open Day at Rio Tinto’s copper mine at Northparkes NSW (near Parkes) on March 2 2013. This visit was an opportunity to experience and understand some of the robots and other actors around the robots. These actors smoothed and accelerated the movement of ore from 600m underground to the surface, onto stockpiles, and finally on train to the ports.
Among the robotic actors on this site was the Loader Hauler Dumper (LHD) from Swedish manufacturer Sandvik. This vehicle is a hybrid operated / autonomous / remote operated rock mover. It is a robot that navigates its load from draw points deep underground, and brings it nearer the surface for crushing and refining.
As it turns out, the LHD is only one among many kinds of robot actor, changing the mine’s technological shape. Mining is slowly changing from being a series of discrete tasks by different actors. Each smoothing works towards turning the mine into a continuous process with greater ongoing measurement and control, in the name of efficiency (continuous mining is on the long-term agenda for many miners).
The story is not that the mine contains robots — it is the whole mine itself that is becoming robotic. More and more components afford remote sensing, feedback and continuous control. Surveillant components (cameras, sensors, robot mounted cameras and so on), offer miners various kinds of agency that bring into play more consistency in managing control flows. The flow of crushed copper ore takes the ore to the stockpile measures the ore as it passes on a weightometer.
The conveyor belt takes crushed ore to the stockpile
Beyond the Northparkes itself, Rio Tinto has an eye to the future. In the Pilbara WA, it has introduced the enormous robotic dump trucks, autonomous drills, and soon autonomous trains. The robotic mine of the future is being built one component at a time, motivated by deeper ambitions of efficiency and control. For now, miners’ bodies and minds remain the dominant actors in most mining practices. The inspiration for efficiency in the ‘Mine of the Future’ operates as a present guiding vision as both internal mantra and PR rhetoric.
The vision of a mine without humans on site is, perhaps, compelling for many. Certainly it allows management to control. Many workers prefer the conditions of remote operations and control centres. Some external observers see the value of this change. Human bodies are clearly outside their element when digging up elements. Bodies are inherently vulnerable in underground environments, and in the presence of massive machinery and explosives. Safety is the mine’s ubiquitous guiding force. Miners’ flouro jackets and safety helmets are a uniform for those avoiding risk.
The body’s capacities to complete tasks repeatedly, and precisely, are also limited, in comparison with many emerging devices. However, the introduction of new devices is quite uneven. On the site, hi-tech gear sits alongside traditional tools. The mine uses up-to-date monitoring systems alongside a tag board at the mine entrance. Each miner must post their tag onto the appropriate spot on the Surface Tag Board when they go underground. Until all the tags are accounted for, there will be no blasting.
Safety serves a double role, imposing control over risky situations, and justifying greater control over miners’ actions. At one level, mining control regimes are undoubtedly justified by the high level of risk. An accident in this mine in 2003 killed four workers (Hebblewhite 2003). On the other hand, Danger is management’s collaborator, justifying tighter control over the workforce. The logic of the safety / surveillance pair is gradually bringing to mine sites a regime of control (Deleuze 1992). Remote systems, feedback, and constant training of workers is less the mode of surveillance from outside, and more control over thresholds of movement.
The risk of deviance is tripled when the possibility of surveillance, the actual risky environment, and the technologies placing the worker under control combine. Control displaces and reconfigures the labouring body for as long as it takes to remove the bodies from risks. When explosives are involved there is no option but to remove the workers bodies from the the location.
Dynamite is a 150 year-old technology that introduced non-human force of explosives to reduce hand-digging. The technique of block cave mining used at NorthParkes is an efficient (but not particularly safe) technique that uses explosives to create massive rockfalls underground. These funnel the fractured ore into draw-points, leaving the ore exposed, but in the dangerous location under rockfall.
The showcase of the site is the Sandvik Automine LH514ELHD: a bright orange vehicle with a large scoop at the front. The vehicle can be controlled remotely from the surface. It also features laser scanners and intelligence that allow it to take control of the vehicle to follow a trail towards the surface. This remote-controlled and autonomous system was considered a trail-blazing implementation in 2010. These new technologies remove operators people from the most dangerous places, and returns them to a more controlled environment.
Becoming the load in a Sandvik Loader Hauler Digger (LHD).
The robotic components: laser scanners guide this robot vehicle by following dead-reckoning tags deep underground, guiding the LHD.
This installation at Northparkes is strategically important for the Swedish company that produces this vehicle. The Australian mining environment is dominated by Caterpillar. It is also part of rapid changes in mining that are withdrawing human bodies, and into control rooms. These are changing the profile of workers, and possibly jettisoning those who don’t have the right profile of expertise.
Rio Tinto’s open day is itself a form of smoothing, building relationships, and removing the potential obstructions in public opinion or expectations of potential employees. Rio Tinto is very active in controlling perceptions of the company. They produce an array of reports, websites, media releases and videos. For example, ‘The Miracle of Copper‘ offers an award-winning, company friendly account of the processes of copper mining. Using the latest vehicles: LHDs and open days; training and public videos; websites and conference presentations — Rio is communicating the many of the values of Rio’s ‘Mine of the Future’. The company has extended their regimes of control away from disciplined secrecy (such as in Ok Tedi) and towards smoothed operations of PR and automation.
Deleuze, G. (1992). Postscript on the Societies of Control. October, 59, 3–7. Hebblewhite, B. K. (2003). Northparkes Findings – The implications for geotechnical professionals in the mining industry, 1–8. (see links)
Rio Tinto (2010) ‘Ore processing’ Northparkes website http://www.northparkes.com.au/ore_processing.aspx
Here is the abstract for my paper at the 2012 Cultural Studies Association of Australia conference. I presented it on December 5, 2012.
Mining automation, displaced labour and materialities of communication
Digital Cultures, University of Sydney
Information systems, remote operation and robotics are currently being introduced into mines around the world. As miners reconfigure communication, control and labour, mining practices that have barely changed in a century are being transformed. This paper analyses innovations such as remote operation of mining, and autonomous systems as media changes, as well as changes in labour processes. The paper follows in reverse the historical arc of Harold Innis, who began in geographical economics (cod, fur, railways in Canada) before pioneering a materialist, longue durée historical media theory.
Mining is among the most basic material human practices. The blasting, loading, hauling, processing and shipping of iron ore is a rudimentary process performed on a huge scale. Digital systems don’t immediately change these material practices, but introduce new information and control flows. The autonomous Komatsu trucks now hauling ore in the Pilbara are little different physically from the human-driven fleet, but afford a precision, continuity, and smoothness of operation that human drivers could not tolerate. Digital media are valued in mining for their greater ‘efficiencies’, and their centralising and visualisation of monitoring and control of mine sites, which can be thousands of kilometres apart. These changes in machine/material communications and autonomy have implications for the kinds of work, the kinds of workers, and the kinds of communities that can cooperate with the mines, and many other workplaces, of the future.
Robot searching in belief space: field robots and their contingent encodings of unknown environments [CODE Abstract]
CODE conference: A Media, Games & Art Conference, Swinburne – 21-23 Nov 2012
Robotics research since the 1980s has been establishing codes, conventions and practices that are likely to govern a generation of autonomous robots that is becoming ready for the field. Today’s engineering choices will define the domains of possibility for robots that will inhabit domestic, public and professional spaces in the future. Among their distinctive features are algorithms that degrade gracefully to allow robots to act in environments that they do not fully ‘understand’.
Field robots are distinguished from industrial robots by their capacity to sense, encode and move around unfamiliar spaces. If robots are a kind of medium, their defining features are their capacity to sense and measure new spaces autonomously, identify salient features, and calculate optimal pathways to move and act. The ‘optimal’ pathways calculated by on-board sensors are necessarily imperfect, but because the robot is a physical entity, its agency must always be recoverable. In one engineering approach to this problem of imperfect information, the robot is said to translate space using ‘heuristic search in belief space’ (Bertoli & Cimatti 2002), where belief space is a kind of formalism of contingency opening onto a certain uncertainty. There is a poetry in engineering discourses as they grapple with the unpredictable and the infinite.
As autonomous technical actors are able to adapt to unpredictability, they themselves become less predictable, moving from striated to smooth spaces (Deleuze & Guattari 1987), and from a high degree of control characteristic of simulation to using systems of code that are adaptable to constant adjustments and compensations. Unlike the GUI of personal computers, robots will not necessarily present users with interactive interfaces. Instead, the robot has its own parasocial integrity and autonomous. However, the conventions for relationships with human actors sharing the same physical and social spaces as field robots have yet to be clearly defined.
This paper will explore these ontological and ethical questions about the operation of code in the world as manifest in field robots.
Notes for Chris Chesher on ABC Northwest (Karratha)
September 3, 2012.
At 1030am I talked with Cristy-Lee Macqueen from ABC Northwest.
Mine sites are changing, as robotic technologies are taking on communication and control roles previously held by people. These changes have been coming for some time, but there has recently been a shift from trialling autonomous systems towards using them in production.
In 2008 the first autonomous trucks were first introduced experimentally, carrying waste products at Rio Tinto’s West Angelas mine. The trials seem to have been a success, as the five Komatsu autonomous trucks covered 570,000 kilometres over 897 days at work between them until February this year.
The old model: Komatsu 830 with human drivers.
In late 2011, the autonomous trucks were reassigned, entering the iron ore production process along with five new trucks, hauling ore at the Junction South East pit of Rio’s Yandicoogina mine.
These ten trucks will undoubtedly be joined by more new autonomous trucks. Rio Tinto reached an understanding with Komatsu in Novermber 2011 to buy 150 Komatsu Autonomous Haulage System trucks over the following four years. It’s not clear what the impact of the iron ore price slump will be on these acquisitions, or how they will fit into Rio’s overall processes.
Komatsu documents that these imposing trucks are fitted with a range of sensors that allow them to operate very safely and accurately. They use laser, radar, GPS, and communications systems to help follow a digital map of the mine site with a lot of precision. The trucks are coordinated by Rio’s control centre 1500 km away, in Perth.
In addition to these developments, Rio has committed over $400 million to automating trains over the next few years. Other parts of the mining process, such as drills, are being automated, or being tagged with location beacons.
Safety is one of the motivations for introducing autonomous systems. A driverless vehicle can’t injure the driver. Autonomous systems don’t have lapses in attention, or drive erratically.
Another reason is to increase production efficiency. Autonomous trucks don’t take breaks. They don’t need to work in shifts. Together, these autonomous systems can work towards the goal of continuous production, where the mine produces an uninterrupted stream of ore.
I’m an academic at the University of Sydney. I am here in Karratha trying to get a sense of how people in the Pibarra feel about the changes to mining work as mining automation is introduced. I’d appreciate if anyone with experience or opinions about mine automation to call in. I’m recording this program, and I’d like to use the transcript in my research. You can find more about my project on my blog http://followingrobots.wordpress.com
Whether these goals of safety and efficiency are achieved, it seems likely there will be changes to the experience of mining. It may affect the social life of mining towns.
To bring up a very different example, when mobile phones became available, they seemed at first to be just a phone you could carry around. In fact, they were quite different from fixed phones. They allowed people to change the way they organised their lives. Rather than make detailed arrangements ahead of time, people with mobiles could easily change plans at the last minute. With smart phones, people could make images and change them, making their own media.
Of course, an automated mine is very different from a community of mobile users. The control centre (opened in 2010) gathers detailed information across several mine sites, centralises control, and provides a place for collective expert decision-making. Remote operation allows operators to take over some stages of production, and allows a small number of people to control many machines. The mine site increasingly becomes a rationalised, controlled and regulated rock factory.
Advocates point to potential benefits of automation for workers. It can take away dangerous, dull and dirty work that nobody wants to do. Mine automation may reduce risks of injury and death. By reducing workers on site, it may reduce fly-in-fly-out work, allowing expert operators to work in urban control rooms. This may take social and economic pressure away from remote mining communities. See also BAEconomics Report.
But there are some potential draw-backs: some people may lose their jobs to autonomous systems, and these changes may raise industrial pressures. The high degree of control over mine sites may be extended to new expectations for those working alongside autonomous systems. The dependence on planned communications systems and GPS guided technology may bring some fragility to autonomous operations, in comparison to the more resilient and adaptable human operated systems.
The long term implications of large scale use of autonomous systems are yet to be revealed. As WA will soon host the largest fleet of autonomous mining vehicles in the world, the unanticipated implications, and the qualitative shifts in mining practices, are likely to play out here.
If you have experience or opinions about mining automation, please leave your comments below. I may use these comments in my research.
Recently I presented a paper called ‘Materialising robot platforms’ on the affordances, environments and networks of three Korean service robots. The topic of my paper was something of an outlier in a conference called ‘Platform Politics’ at Anglia Ruskin University, Cambridge, organised by Jussi Parrika and Joss Hands.
Most other papers identified either with political theory and technology, or with platform studies: analysing how the underlying technological infrastructures play out in fostering certain social and political outcomes. My paper was closer to the latter category, examining in particular some of the political implications of technological artefacts: the placement of sensors and motors in robots that respond to touch, allow remote teaching, and bow to indicate subservience.
The conference was video recorded in a pretty rudimentary way using UStream. It is pretty hard to follow the paper from this video. The abstract is below (although of course this doesn’t really reflect what I talked about).
Chris Chesher Research and development in robotics is currently developing a range of network-connected material platforms. This practice is producing robots increasingly tuned towards particular lifeworlds: language teaching robots in classrooms; service robots in public spaces; container-handling robots in ports; rescue robots in earthquake zones, and so on. These specific platforms diverge significantly from the general-purpose robot of popular imagination as robots are made increasingly real as they are themselves formed by their multiple attachments across physical, social and institutional spaces. This paper draws on recent interviews with researchers at the Australian Centre for Field Robotics, and company representatives at the Robotworld tradeshow in Korea. The interviews examine the rhetoric and practices by which robot platforms are increasingly blackboxed as technical innovations in ways that are informed by narratives of the application environments, and strategic connections with institutional networks. A robot platform is constituted by a singular combination of elements: sensors, operating systems, programming and effectors (motors, screens, speakers, etc). However, these components must work together towards creating a robot that can perform as an autonomous
actor in forming relations within specific environments. In talking about the robots, engineers, developers and salespeople often provide rich narratives featuring the robots in particular physical and social environments. Developers are also aware of the institutional connections in operation that will be crucial in securing the robot’s current and future existence. The Korean company Dasarobot’s English language teaching robot must capture the interest of teachers, but outside their direct affiliations with schools. Development communities are establishing core features of contenders for future robot platforms, abstracted below the level of particular applications. For example, many robots use similar autocharging systems to respond autonomously to the common problem of a low battery. Some robots use custom operating systems, while others use open source ROS such as those from Willow Garage and Microsoft. The range of issues in robotic platforms gives the problem of software platforms a material base, as seen in the collaborations and conflicts between key mechatronics disciplines of software engineering, mechanical engineering and electrical engineering. Meanwhile, as robotic platforms stabilise, there are increasing enrolments of other disciplines: media art; media practice; performance; design; marketing; cinema and so on.