About machine capabilities versus human sensitivities

For Recognition, an artificial intelligence program that associates Tate’s masterpieces and news photographs provided by Reuters, there are visual or thematic similarities between the photo of a woman with a phrase on her face that reads #foratemer (out Temer) during a protest against a constitutional amendment known as PEC 55 and the portrait of an aristocrat man of the seventeenth century in costumes that denote his sovereignty and authority. In times when intelligent and thinking machines, like chatbots, are a topic widely discussed I wonder if the algorithms that created the dialogue between these two images would be aware of the conflicting but no less interesting relationship between resistance and power established between them.

Visualizing time, texture and themes in historical drawings

Past vision is a collection of historical drawings visualized in a thematic and temporal arrangement. The interface highlights general trends in the overall collection and gives access to rich details of individual items.

The case study examines the potential of visualization when applied to, and developed for, cultural heritage collections. It specifically explores how techniques aimed at visualizing the quantitative structure of a collection can be coupled with a more qualitative mode that allows for detailed examination of the artifacts and their contexts by displaying high-resolution views of digitized cultural objects with detailed art historical research findings.

Past vision is a research project by Urban Complexity Lab at Potsdam University of Applied Sciences.

Reference: “Past Visions and Reconciling Views: Visualizing Time, Texture and Themes in Cultural Collections.” ResearchGate. Accessed March 8, 2018.

Visualizing cultural collections

Browsing the content from Information Plus Conference (2016 edition) I bumped into a really interesting presentation regarding the use of graphical user interfaces and data visualization to support the exploration of large-scale digital cultural heritage.

One View is Not Enough: High-level Visualizations of Large Cultural Collections is a contribution by the Urban Complexity Lab, from the University of Applied Sciences Potsdam. Check the talk by Marian Dörk:

As many cultural heritage institutions, such as museums, archives, and libraries, are digitizing their assets, there is a pressing question which is how can we give access to this large-scale and complex inventories? How can we present it in a way to let people can draw meaning from it, get inspired and entertained and maybe even educated?

The Urban Complexity Lab tackle this open problem by investigating and developing graphical user interfaces and different kinds of data visualizations to explore and visualize cultural collections in a way to show high-level patterns and relationships.

In this specific talk, Marian presents two projects conducted at the Lab. The first, DDB visualized, is a project in partnership with the Deutsche Digitale Bibliothek. Four interactive visualizations make the vast extent of the German Digital Library visible and explorable. Periods, places and persons are three of the categories, while keywords provide links to browsable pages of the library itself.

 

The second, GEI – Digital, is a project in partnership with the Georg Eckert Institute. This data dossier provides multi-faceted perspectives on GEI-Digital, a digital library of historical schoolbooks created and maintained by the Georg Eckert Institute for International Textbook Research.

 

Mind-reading machines

A new AI model sort of reconstructs what you see from brain scans.

Schematics of our reconstruction approach. (A) Model training. We use an adversarial training strategy adopted from Dosovitskiy and Brox (2016b), which consists of 3 DNNs: a generator, a comparator, and a discriminator. The training images are presented to a human subject, while brain activity is measured by fMRI. The fMRI activity is used as an input to the generator. The generator is trained to reconstruct the images from the fMRI activity to be as similar to the presented training images in both pixel and feature space. The adversarial loss constrains the generator to generate reconstructed images that fool the discriminator to classify them as the true training images. The discriminator is trained to distinguish between the reconstructed image and the true training image. The comparator is a pre-trained DNN, which was trained to recognize the object in natural images. Both the reconstructed and true training images are used as an input to the comparator, which compares the image similarity in feature space. (B) Model test. In the test phase, the images are reconstructed by providing the fMRI activity of the test image as the input to the generator. (Shen et al, 2018)

Check the Journal Article here

Reference: Shen, Guohua, Kshitij Dwivedi, Kei Majima, Tomoyasu Horikawa, and Yukiyasu Kamitani. “End-to-End Deep Image Reconstruction from Human Brain Activity.” BioRxiv, February 27, 2018, 272518. https://doi.org/10.1101/272518.