Abstract
This article explores hybrid digital/physical workflows in the building trades, a high-skill domain where human dexterity and craft can be augmented by the precision and repeatability of digital design and fabrication tools. In particular, the article highlights two projects where historic construction techniques were extended through live motion capture of human gesture, information-rich visualization projected in the space of fabrication and custom robotic tooling to generate free-form running moulds. The first case study explores decorative plastering techniques and an augmented workflow where designers and craftspeople can quickly explore patterns through freehand sketch, test ideas with shaded previews and seamlessly produce physical parts using robotic collaborators. The second case study reimagines a roman vaulting technique that used terracotta bottles as part of an interlocking masonry system. Motion capture is used to place building elements precisely in material arrays with real-time visual feedback guiding the hand-held placement of each bottle. These case studies serve to underscore the emerging importance of reality capture in the design and construction of the built environment. Increasingly, the algorithmic power of computational tools and the nuances of human skill can be combined in hybrid design and fabrication workflows.
Keywords
Motivation
The human body fosters a wealth of tacit knowledge vital to cultural, political and economic dimensions of human life. Think of the learned dexterity of a surgeon’s fingers, the buoyancy of a dancer poised to leap or the deftness of experienced hands guiding a chisel through natural wood. Despite the body’s centrality to many important modes of human endeavour, technology has often replaced bodily skill with mechanized production. In the architectural arena, industrial design and manufacturing tools have significantly altered the relationship of human craft to the design and production of the built environment. The expressive skill of the craftsperson and designer are at risk when we consider the innumerable hours of design creativity forced to flow through an architectural intern’s mouse finger or the mind-numbing repetitiveness of supervising most industrial manufacturing equipment. This article outlines two case study projects exploring these themes. The projects were developed as part of ongoing research at the Carnegie Mellon School of Architecture’s Architectural Robotics Lab.
Human–robot collaboration in high-skill domains
Despite the prolonged history of displaced human skill in relation to industrial production, this need not remain the case. Contemporary developments in robotic fabrication and real-time sensing are disturbing the equilibrium of the design and industrial manufacture of architectural building components. This article outlines the development of digital tool-sets to enable collaboration between humans and machines in the high-skill domains of the building trades. An intuitive interface using live motion tracking (Motion Capture (MOCAP)), digital projection and gesture recognition was created to move designers and craftspeople away from offline programming at a dedicated computer to using tracked gestures, hand-tools and building materials to interact with digital models and robot collaborators. This entailed connecting the disparate worlds of constraint-based motion planning and computer-aided design (CAD)-generated offline programming. Motion planning excels at open-ended indeterminate tasks and real-time decision-making. Offline programming, often favoured by designers, allows for robust geometric construction and visual feedback during the design process.
Augmented fabrication contexts using reality capture
The domain of reality capture is quickly changing the design and fabrication environment. 1 The ability to extract precise measurement and visual information from the physical environment in real-time is collapsing the space between representation and reality in architectural construction. Reality capture bridges the gap between the geometrically idealized world of design representation (e.g. construction drawings) and the messy, non-standard world of as-built construction. The emerging space of design and fabrication is fully synthetic where the virtual and the physical are equally real partners in navigating the translation from design to construction. In this context, human gesture re-emerges as a vital medium to harness our creativity and skill.
As computation becomes more ubiquitous and embedded in our physical environments, many high-skill domains have become hybrid in nature, leveraging the precision and repeatability of computational tools along with the dexterity and expressivity of human skill. Whether competing at Chess or conducting surgery in the operating room, hybrid workflows are often more effective than tasks performed by humans or computers acting on their own. 2 The building industry has been late adopters of robotic technologies. Although the building industry has borrowed heavily from the manufacturing sector encouraging many industrialized building materials and construction methods, the primarily autonomous robotic assembly approaches found in many factories are often ill-suited to the challenges of on-site construction and the degree of customization in architectural production. 3 The factory favours automation, which requires the reduction of complex tasks to their lowest common and repeatable denominator. The shop and construction site, however, favour the efficient and varied application of dexterous skill to open-ended tasks. The logistics and efficiencies of the factory floor cannot simply be re-created in the shop or on the construction site. Advances in contemporary technology suggest promising possibilities for human–machine collaboration in the building trades. Three types of technology central to these advances in fabrication settings will be demonstrated in the case studies of decorative robotic plastering and Roman masonry techniques:
Real-time sensing. Human–machine collaboration in fabrication settings relies on the exchange of information based on actions in the physical world. Capturing human gesture for skill-transfer, tracking tools precisely in three-dimensional (3D) space and sensing material behaviour in response to computer numerical control (CNC) tooling are all made possible through real-time sensing.
Information-rich projection mapped to physical context. The typical shop is littered with devices used to bring a sense of measure to the physical context of fabrication. Tape measures, fixtures, jigs, clamps and templates allow material and tools to be used with intention. Information has been traditionally passed from paper drawing sets through these physical mechanisms to produce reliable results. The advent of projection mapping allows for a seamless transfer of information from digital models into the physical space of fabrication. Measuring grids, custom layouts, part labelling and assembly information can all be projected onto the 3D surfaces of the shop in real time.
Seamless incorporation of custom robotic tools. Industrial robots encourage custom tools and workflows for shaping a wide variety of materials. Unlike dedicated machine tools, robots offer more flexibility when adapting to the diverse demands often placed on architectural fabricators and construction crews by the custom requirements of specific projects. Robots become part of a flexible infrastructure that can support customization and invention.
Case study I: running moulds, an exploration of architectural plaster
This section describes the traditional craft of plaster running moulds and the relevant constraints considered in robotically augmenting the process in contemporary design practice. The running mould is an ancient plastering technique used to construct ornamental architectural elements in interior and exterior applications (e.g. cornice moulding). 4 The technique involves cutting a decorative profile into a piece of sheet metal and running the profile repeatedly over multiple gauges of plaster as it cures from liquid to solid. Although running moulds are often visually combined with other plastering elements, the fabrication process is distinct from plastering techniques that require casting into dedicated moulds (e.g. dentil moulding). Running moulds can be prefabricated in a tradesperson’s shop or run in situ on a construction site. The technique requires an experienced set of hands for appropriate setup and execution during plaster’s rapid curing process. The key constraint is to run the profile across the curing plaster with as close to identical passes as possible. Because repeatability is essential to a smooth finish, the metal profile is mounted to a sled, which is typically guided by a set of rails or a fixed pivot. Human dexterity is required to smoothly move the profile across the curing plaster, but the overall path of the profile and the pressure shaping the plaster finish are controlled by the guides. Being constrained by physical guides when running profiles by hand dictates a fairly limited set of possible running mould geometries (e.g. straight lines, circles and ellipses). Deviation from these shapes requires costly and time-consuming setup of custom guides.
Custom profiling tools were developed for a six-axis industrial robot to test new possibilities for the running mould technique.1,5 Because all offline robot motion planning is reliably repeatable and six-axis robots have a high degree of kinematic freedom in space, physical guides became unnecessary. This entailed the ability to explore free-form moulding in complex 3D configurations, producing decorative profiles that would be difficult, if not impossible, to construct by hand (Figure 1).
Morphfaux decorative robotic plastering on free-form surfaces.
The workflow outlined in this article responds to the possibilities afforded by contemporary technology to augment traditional building trades. If the logical layout technique for traditional moulding design required ruled drawing, using straightedge and compass, then the kinematic freedom afforded by robotic technology suggests a more direct connection to freehand sketching. The following sections illustrate how bodily skill can remain central to designers and tradespeople when exploring the augmented space of robotic construction through seamless tangible interface in fabrication settings.
Workcell setup
The following section describes the physical workcell developed around a plastering table for producing running moulds and the software developed to combine real-time sensing, projection mapping and robot motion control in a hybrid fabrication setting (Figure 2).
Physical workcell with MOCAP cameras, ceiling-mounted projector and industrial robot.
A six-camera motion capture array is mounted above a 1.2 m × 2.4 m work table. Real-time tracking of a stylus allows users to explore the constraint space of decorative plaster moulding patterns through freehand sketching on the tabletop. A custom streaming component in grasshopper, developed by a collaborative team for a workshop at the 2014 Robotics in Architecture conference, brings hand sketching information directly into the CAD environment. 6 A custom calibration script automatically aligns all tracking with the lab robot’s base coordinate system. Once calibrated, other tools or work objects can be tracked relative to the robot’s coordinate system (Figure 3).
Real-time tracking of stylus with projection of drawn curves on work surface.
Visual feedback is projected onto the work surface using a ceiling-mounted wide-throw projector. Projection mapping is calibrated with a custom Python script in grasshopper using hand-placed MOCAP markers. From the CAD environment, design and assembly information can be projected onto the work table, including shaded previews of potential profiles, start and end point locations of neighbouring pieces in a pattern, and underlying grids for measure. An interactive dashboard is also projected onto the tabletop and allows users to explore different profile curves and negotiate the part to whole relationship of individual tiles in the overall pattern (Figure 4). The user can move fluidly between design iteration with the stylus and parametric pattern generation displayed through the projector.
From right to left: projection of dashboard on work surface, projection of fabrication information prior to drawing, projection of shaded preview and final plaster sample from drawing.
As the pattern develops through iteration, the user can generate drive curves for robotic plastering on the work surface directly from the stylus seed input. Users can choose to smooth the raw input to remove or amplify noise from the hand-drawn mark. Robot motion control is also handled within Grasshopper using the HAL plugin. 6 Custom Grasshopper scripts help with motion planning in order to avoid joint errors from the input of hand-drawn guide curves. A custom end of arm tool allows for plaster profiles to be quickly interchanged (Figure 5).
Custom end of arm plastering tool with interchangeable profiles.
Demonstration
A sample pattern was tested to demonstrate the potential of augmented fabrication workflows in architectural plaster. The authors developed a parametric patterning script that generated radial arrays around a group of asymmetric obstacles commonly found in a contemporary ceiling plane (e.g. light fixtures, columns, HVAC (heating, ventilation and air conditioning), fire suppression). Historically, plaster rosettes were installed around chandeliers in carefully symmetric patterns. Users seed the algorithmic pattern generation with two curves, sketched using the tracked stylus. An inner and outer drawing boundary and a measuring grid are projected at one-to-one scale on the work table to aid in hand sketching. Once the two seed curves are generated, users can see a shaded display of the full plaster profile projected around the ceiling obstacle. Users can explore the parameters of the pattern, changing the density and spacing of each input curve. Users can also iterate the input curves with further sketching, getting visual feedback in real time (Figure 6).
Pattern generation and exploration using hand sketching and parametric design.
In this example workflow, the virtual status of representation, which typically precedes construction and is constrained to screen or paper space, permeates the physical space of construction. The collapse between contexts for representation and contexts for making allows for haptic exploration of design constraints that are both materially pragmatic and algorithmically extensive (Figure 7). Rather than replacing human skill through automation, human gesture expresses itself at the intersection of physical and digital design processes, augmenting what is possible in the high-skill domains of the building trades.
Sample pattern plastered from hand-drawn input.
Case study II: roman vaulting techniques
Tubi Fittili refers to an ancient Roman masonry vaulting technique. The literal Latin translation means clay tube (Figure 8), and the process involves using clay bottles interlocked by mortar to form large-span arches. The technique provided an economic solution to many civic and infrastructural typologies in the Roman Empire due to its easy assembly without the use of complex wooden false work.
Example terracotta bottle with MOCAP markers.
This proved especially beneficial in regions of the empire where timber was extremely scarce. The inner and outer surfaces of the interlocking masonry units were often covered with mortar or finish plaster.
Method constraints
Given the constraints of historical surveying techniques, a limited world of forms, based upon the barrel vault, was possible using the clay tube technique. Often vaults were started from an arc traced on an end-wall and checked along the length of the barrel using analogue surveying equipment. Basic intersections between vaults could also be achieved to combine programmatic volumes. 7 Individual clay tubes varied in profile, but many flared on one side to allow nesting with neighbours at a variety of angles. Thus, despite a great deal of potential variation being afforded by the part-to-part relationship of masonry units, the overall system was dictated by the global constraints of easily measurable forms.
Augmented surveying techniques
Using MOCAP to track building components in a fabrication setting opens the possibility of exploring the affordance of parametric modelling to generate non-standard arrays from a simple, repeating part, in this case a terracotta masonry unit. Rather than letting global form constrain the system, the full range of part-to-part relationships can be explored using more complex drive surfaces.
Exploring the constraint space of possible forms entails developing algorithmic approaches to generating new forms populated by bottles and addressing the challenges of constructability of non-standard forms. The following sections will discuss the development of these two issues.
Generation of bottle arrays
In the first case, custom scripts were developed in GHPython to populate parametric drive surfaces with interlocking arrays of bottles (Figure 9). These scripts resolved issues of unit spacing, intersection, gap filling and choosing the shortest effective span across a given surface. The script can quickly accommodate changes in bottle geometry to allow for design choices in the building unit to propagate across the array. A basic user interface allows fabricators to select target bottles for placing in the array and cycle through rows and bottles within the dimensions of the array.
Generation of target model from GHPython script.
Bottle placement in physical scenes
Based upon the algorithmically generated target assembly, a motion tracking interface allows users to precisely place physical bottles in the environment and check them against the target model for locational accuracy. Simple visual cues projected into the fabrication scene indicate how close a target bottle is from its intended position. Bottles are indicated as placed when they fall within a user-defined positional tolerance (Figure 10).
Live motion tracking of bottle compared to virtual target.
Future iterations will also include an interactive dashboard where users can change target bottles and rows and see other relevant construction information using human gesture. Having tested the accuracy of motion tracking in a fabrication environment, the positional accuracy of each bottle will be well within the construction tolerance of the vaulting system. 1 It is also worth noting that when assisted in placing the units, the human hand can position construction elements in 3D space with an accuracy well within the construction tolerance of many material systems. Depending on the number of cameras in a MOCAP array, the capture volume can be viable at an architectural scale. It is also possible to reposition and recalibrate the cameras during the construction process if the vault exceeds the bounds of the MOCAP field of view. Potential benefits of using MOCAP for surveying in fabrication settings include the high level of accuracy of a properly calibrated system; relative affordability compared to other laser scanning systems; the ability to capture hand-tools, human gesture and building components in real-time; and the extendibility of the capture volume. Possible drawbacks include less range than most laser scanning techniques and a greater sensitivity to ambient light conditions and reflective materials in the scene.
Conclusion
The two case studies mentioned in this article illustrate how computationally driven machines and human skill can be combined in collaborative contexts using reality computing. In both cases, emerging reality computing technologies are leveraged to find new potential in the design and fabrication of the built environment.
Future developments
Future developments of case study I will include developing a Kinect-based gesture interface to seamlessly navigate dashboard settings while carrying out fabrication tasks in a shop environment. We would also like to develop a robot-mounted projector where information can be mapped to any surface within the robot’s workcell. Posing the robot for specific projections would allow for a larger and more flexibility projection environment. Finally, the team would like to develop an automated material delivery mechanism, integrated with the end of arm plastering tool. Currently plaster is distributed by hand, based on visual prompts from the projector. Automated delivery could be more efficient and require less material waste in the initial passes.
Future developments of case study II will include the implementation of an interactive dashboard to control the target bottle and row, that is, testing robot-mounted MOCAP arrays to seamlessly reposition cameras in the scene based upon desired capture volume. In this scenario, the robot would serve as surveying apprentice, that is, testing the system at half scale with a physical prototype.
Importance
There are many areas where hybrid fabrication environments can augment the building trades. Emerging technologies in robotic fabrication and real-time sensing enable small-scale production, afford a higher degree of customization and allow designers greater access to the means of building production. The difficult relationship between the human body and industrial machines can be reformulated in this new context. Ultimately, a collaborative relationship can emerge where the salient characteristics of human skill and machine precision work in tandem towards augmented paradigms of fabrication. Working towards robot–human collaboration in high-skill domains entails defending the sustained, often pleasurable, mind-and-body-work required for architectural design and production. As educators and design professionals, the body’s capacity to explore through sketch and material play can and should remain central to the design process. Old distinctions between analogue and digital design processes no longer need jeopardize this possibility. In the building trades, many jettisoned techniques can be reframed with new potential. When humans and machines collaborate in high-skill domains, we can look to the past to inform the future.
Footnotes
Acknowledgements
HAL Robot Programming and Control was used to stream MOCAP sessions into Grasshopper. Steven Mankouche is Assistant Professor in Architecture University of Michigan, Collaborator on the Tubi Fittili project and creator of the test bottles. Dr Garth Zeglin is with Carnegie Mellon’s Robotics Institute.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
Seed Funding was provided by Carnegie Mellon University, Integrative Design, Arts, and Technology (IDeATe) for the project title ‘Human Machine Virtuosity: building bridges for accelerated digital-physical iteration’. Garth Zeglin was the co-principal investigator (2013–2014).