"Data dust … used for algorithmic electioneering to enforce authoritarian, anti-democratic policies." ZONTA Award - Jury Statement, 65th Kurzfilmfest Oberhausen
A HipHop Comedy against automated propaganda, directed by Manu Luksch
Starring: Gunman Xuman, Lady Zee, OMG
Convenience, not choice. Efficiency, not freedom. Frictionless experience.
In marketing and the retail sector, data analytics is widely used to profile and micro-target consumers and to predict behaviour. The ultimate goal, apparently, is for humans to be able to outsource all decision-making to machine intelligence (make Google do it!). What is at stake within the political realm?
ALGO-RHYTHM , shot in Dakar with the participation of leading Senegalese musicians, poets and graffiti artists, probes the insidious but comprehensive threats to human rights and agency posed by the rise of the quantification and algorithmic management of daily life. Using hiphop, drama, street art and data-driven filmmaking, the work explores how our embrace of the convenience of machine intelligence, refracted through the slick interface of smartphone apps, makes us vulnerable to manipulation by political actors. Recognising the urgent need for a new visual language to illuminate this concern, Manu Luksch collaborated closely with Jack Wolf and Mukul Patel to develop a hybrid narrative form that unites photogrammetry and volumetric filmmaking with traditional approaches. Through its auratic and poetic use of computational imaging technologies, ALGO-RHYTHM scrutinizes the limitations, errors and abuses of algorithmic representations.
Directed by Manu Luksch
Starring: Gunman Xuman as Mr X, CEO of Data Analytica, and Mr Y, Good News anchor
Lady Zee and OMG as the Presidential Candidates
Featuring: Adja Fall, Illégal, Kruh Mandiou Mauri, MC Mo and Rhapsod.
Lyrics based on a text by Manu Luksch & Mukul Patel, co-written with Gunman Xuman and Keyti Cheikh Séne
Live Graffiti: Madzoo
Immersive FX consultant & recording: Jack Wolf
Animation: Georgi Stamenov
Camera Op and dronista: Malick Sy
Production Management Dakar: Baila Sy
Project Assistant Dakar: Jojo Shone
Supported by the Open Society Foundations and denkbar projektentwicklungs gmbh
“Data dust, rushing data points: convincing images for the fine-meshed global system of voluntarily given, retrievable data … used for automated propaganda campaigns and algorithmic electioneering to enforce authoritarian, anti-democratic policies.”
ZONTA Award – Jury Statement, 65th International Short Film Festival Oberhausen 2019, International Competition. Germany
Interview with Manu Luksch, Substrand, 01/2019
Voters await offcial result in Senegal’s presidential elections, Aljazeera (video), 02/2019
About the method
Multiple approaches, tools and techniques were chosen to reflect the multilayered content of the work. A major part of the imagery was created through photogrammetry, a method of ‘computational photography’ in which algorithms are used to calculate the path of the ray that hits the image sensor. Our use of computational photography to reveal previously hidden information in an image points to the use of Ai on personal datasets to reveal previously hidden information about individuals or groups.
The computational photography element of the work was carried out using low-cost tools and DIY/hacker approaches and custom-programmed effects as far as possible, to demonstrate that in the ‘AI era’, the incumbent powers do not have a monopoly on computation. Our image capture equipment included a DSLR, a pair of compact video cameras with 235° fisheye lenses mounted back-to-back to capture a complete 360° field of view, a drone-mounted 4K video camera, and a Microsoft Kinect motion sensing game input device.
Stills from the DSLR and sequences of stills from the 360° immersive video and the drone camera were fed into photogrammetry software that stitched them together and computed depth information, to create a large 3D point cloud or polygion mesh. The Kinect’s time-of-light’ camera illuminates nearby objects with pulses of IR light, and calculates a depth map (3D image) of the exposed surfaces from the timing of the reflected pulse.
The resulting motion and depth information is used to construct point clouds (a set of points representing the 3D extent of the scene) and polygonal meshes. The Kinect sensor can produce point clouds/meshes at film-like rates, resulting in a a moderate resolution video file that can be ‘moved around’ in three spatial dimensions. Higher resolution static 3D images (of several million points/polygons) are produced from a series of images taken by a moving camera.
These still or motion photograms (points and meshes) can be combined with 2D camera images to create a photorealistic video augmented with depth information (a 3D video, which can be navigated around). We also processed this additional depth/motion data to distort the 2D images in ways that suggest the deduction of additional information, such as the intention or character of a person in the frame. By decimating (simplifying) the polygonal mesh of a person, we extract their essential character – in the same way that algorithms reduce us to a finite set of correlated data points that supposedly reveal who we really are. Facebook – and Visa – know you better than you know yourself. But, like the algorithms managing our societies, our photogrammetric algorithms produce hollow shells of characters…