WHAT A MESS, a tale of collaborative photogrammetry
The following text is a compilation of passages found on the archival website whatamess.city and the result of our latest collaboration between Pedro Gil Farias and I. The archive and associated instalation were first developed during the Realities in Transition residency hosted by V2_Lab for Unstable Media.
PART I | THE PROJECT
For this project we wanted to explore the untapped narrative potential of the urban fabric, through an approach nestled between the practices of plein-air painting, MR interventions and locative arts.
Trading the easel and VR headset for a 3D scanning device (an Apple tablet to be exact), we formed small groups of casual urban explorers, and set out to digitally sample 3D vignettes of the neighborhoods visited over the course of three different walkshops. Although we created our own installation to showcase the 3D captures, we imagined a project which could in turn contribute to further reappopriation.
The residency ended in a Test Lab event, during which visitors were invited to discover the work developed by the residents. For our instalation, we tried to convey to the visitors the sense of awe and discovery we felt as the artists in manipulating 3D scans found online during previous projects like Haul Earth Ledger.
Taking our core project premise as a starting point (exploring the untapped narrative potential of the urban fabric), we 3D printed the three aggregates created during the workshops that preceded the event, and associated them with animated digital twins which became visible (unleashed) when inspected under the tablet.
The experience resulted in a intimate and ambiguous impression that is hard to describe, caused by the realtime manipulation of a thing that is both there physically, but also digitally as well. The feed from the tablet displaying the AR was also juxtaposed using Touchdesigner onto machinima footage captured in Fortnite Creative and Skater XL, two game environments into which we had integrated our 3D captures.
The intent being to emphasize the ease with which such processes could be adopted, aided mainly consumer products (still fairly costly yet rarely used to their fullest potential), and encourage viewers to question what objects, and stories, were used to tell the stories they encountered in virtual worlds.
PART II | THE INTENT
Photogrammetry — From archiving to storytelling
Our main hope for the project is to contribute to the growing body of works that explore the creative and narrative potential of digital asset kits. This corner of the digital arts world, is closely linked to the gaming industry, environmental storytelling, concept art creation, and often leads to incredibly rich questions on representation, narrative biases, or even ad hoc production methods.
In this space, photogrammetry holds a unique position, with gaming studios from EA to PUBG investing considerable resources over the past decade into developing their own 3D scanning pipelines. Since 3D scanning is a process of digitizing elements found in the physical world, many questions quickly arise when thinking of which parts on Earth are digitized and why.
Is it a question of means? Of convenience? Of mainstream interest and gaze? When the use of these assets goes beyond the world of documentation and archiving to enter new narrative worlds, with on one end of the spectrum the Forza Motorsport 7 team turning to the technology to digitize existing race tracks, to the Battlefront team’s captures of the Grand Canyon to populate the deserts of the Star Wars universe, and the PUBG team traveling to Suratthani, Thailand, looking for “abandoned resorts in Southeast Asia” to create its Sanhok map, issues of cultural contextualization cannot be easily dismissed.
As these 3D captures of the physical world make their way into new fictional worlds (worlds often linked to cinematic or gaming blockbuster productions) they can as easily be meaningfully integrated into the work in question as they can be distorted, misrepresented or misappropriated. Being aware of the distortion caused by stereotypes and representational biases in the creation of virtual worlds is a topic Kate Edwards has been a great advocate for, and which makes projects like “Is this the Middle East” incredibly compelling.
“GO BEYOND PHOTOREALISM” claims Quixel on their Megascans page, where they offer libraries of 3D meshes and textures to Unreal Engine users. If one current of practice is to strive for this higher-than-high-hyperrealism, it seemed interesting to us to take the tried and true alternative path to going beyond figurative representation and photorealism: surrealism. A sensibility Oddviz majestically employs in its inventory series, straddling the line between curated archive and cabinet of curiosities.
One aspect that was also interesting for us to explore which paved the way to the name “What a Mess”, is how the inherently flawed aesthetic of 3D scanning — filled with holes, imperfections, non-optimized meshes — echoes the critique that René Boer puts forward in his forthcoming book towards the Smooth City. As a “a highly normative, controlling and arguably oppressive environment, in which gradually all opportunities for productive friction, sudden transitions or subversive transgressions have been eliminated”, the process of smoothing the urban space creates a nice parallel with the gaming industry’s approach to 3D scanning described above: taking it out of context and then optimising it so imperfections don’t show and it runs ‘smoothly’. With that in mind, how can an alternative practice of 3D scanning not only capture the physical qualities of a space when extracting it from the original context, but also capture the contextual traces of the lived experiences of space?
Beyond photorealism — Realism through personal curation
For the scope of this project, we decided to imitate the approach followed by digitization teams in the gaming industry. We hosted workshops, during which participants were invited to craft thematic micro-collections of digital assets by going through the key steps followed by the game industry: scouting locations, capturing assets, and translating them into asset kits.
However, instead of having a game’s narrative and its players’ expectations guide our choice of elements to scan, we instead looked for partners (cultural programers, activists, art festivals) to define the themes. This led us to working with The Public Space Detective, the MOMO Festival, ALL CAPS, to create a thoroughly non-exhaustive yet hyper-personal collections of assets as manifestations of a given topic: The Working City, The (In)Hospitable City, The Porous City, The Annotated City.
Each collection of assets was then cleaned up, curated, and collated into the archive found on this website to be used and reused by other artists should they choose to do so. In doing so, we hope to contribute, however modestly, to the growing wealth of 3D asset kits available online from David Oreilly’s Library of Things to Ali Tufan Ülbegi’s Procreate templates and Kitbash3D’s many thematic urban kits.
The Mimic — Sentient sediment
Finally, our own curation of the assets was inspired by several artists’ works like Oddviz mentioned above, but also Matthew Plummer-Fernandez’s Shiv Integer, a bot (drawing from the Dadaists’ readymade and chance art) which rummages through Thingiverse to randomly pick objects to conjoin into sculptures.
Ourselves borrowing from Dadaist methods we looked into the use of 3D assets in the creation of exquisite corpses, to try and see how we could present make the captures feel alive, as if having a life of their own. Things clicked when we discovered Ranx the Sentient City, a DC comics character, with the power of urban mimicry: “the power to become or mimic the traits of an urban area. Technique of Urban Manipulation. Variation of Superpowered Location.”
Recreating the essence of the mimic: a glitch-like, familiar yet unnerving, sentient-and-larger-than-life-dust-ball, allowed us to showcase the artifacts captured with limited formal manipulation, while giving them a completely unique nature. It also allowed us to echo the critique made by Schuiten & Peeters in their Revoir Paris graphic novel of how pointless and detrimental removing noteworthy artifacts of Paris from their socio-cultural context can be. In their novel, they depict the building of Notre-Dame, a metro entrance, the Arc de Triomphe, preserved in a dark attic, inert, lifeless, in the name of preservation a lot like digital asset kits. This allegory, raises questions of preservation, heritage, cultural contextualization and the “renewed fetish with historical fabrics” that René Boer observes as an element of the smooth city but that also aptly depicts today’s mainstream gaming industry. With AAA games like the Assassin’s Creed series (Notre-Dame, again) that are all too often fossilizing the cultural significance of specific spaces.
The digitization of artifacts, can be a way of celebrating said items by recording their form, their texture, but this celebration is incomplete, and shallow, if not meaningfully contextualized.
By fusing them together in these peculiar creatures we hope to personify this incompleteness while also calling attention to the narrative possibilities (affordances?) that lay in each collection of captures. This got us to stray away from the association with the gaming world and instead dive into the world of geology. There we found exciting parallels to draw with the process of sedimentation, sedimentary rocks, desert roses, resulting in a more mineral aesthetic for our final creations. To close with the words of Matthieux Duperrex in La rivière et le bulldozer (which themselves echo the words of René Boer): “We need, very concretely, to decolonize some of our mineralizations. Living with discordance, and not the flat, homogenous, and sure ground of Cartesian space. […] It is up to us to make sure the verb to sediment […] is not seen as a sinister and mortuary end, but a renewed source of joy.”
