An interactive comic about a blind person and an aphasic person becoming Internet friends

I had a new idea this week. The form of visual images is also a language. The spatial language represented by visual images, long neglected by popular contexts, is an important complement to the temporal language represented by natural language such as Chinese or English. I believe it’s meaningful to use a comic work to tell a story about the relationship between the spatial language and the temporal language. Beyond that, language is often associated with human communication. So I made a blind character Seraphina and an aphasic character Lyra who can only express with images good friends online.

This project will start out as a simple visual novel. The comic will only show chat boxes between the two characters. The typography will be based on the chatroom interfaces, resembling a screenshot of a digital device like a mobile phone or a laptop. I did this because, after some research, I thought the narrative model of using screenshots of conversations directly from chat rooms was very appealing.

And after that, I plan to make it interactive. This interactive comic will be a chatroom site. Users will be able to choose to play either Seraphina or Lyra and experience the character's unique way of exchanging information.

The inspiration board that I created for the visual novel

I did some technical experiments in order to follow up on the future interactivization. One of these is the image annotation feature used by Seraphina, which converts the doodles sent by Lyra into text. I tried three providers that offer an image annotation API: Google Cloud, Azure and Clarifai. I got the following results:

1. Google Cloud requires a credit card bundle, but Vision API has a 90-day free trial. It is not automatically charged when the trial period is over. After some attempts, I got error 503 and could not connect to Google's servers. I don't know if this is a problem with the Google server itself or my geographic location.

2. Azure requires a credit card bundle, but you need to recharge it to use Vision API. i didn't recharge it.

3. Clarifai does not require a credit card bundle. It offers free Vision API usage with limit, which is very friendly for me. After my attempts, it worked successfully. But there is still a small problem that it can only recognize web images, not local images. Clarifai official website: Explore The World’s AI Community | Clarifai
   

A screenshot of my successful implementation of the feature

The aim of my project is to remind people of the importance of visual images as a language and to provoke them to think about the relationship between the spatial language and the temporal language. I am intended to broaden people's choices of how they express their daily messages. At the same time, I want to bring some attention to two groups, the blind and the aphasic. In a way, it is educational.

Edit on 14/4/2024: During the week, I managed to get the visual novel done. This simple prototype will set the foundation style for my subsequent development.

1-3 (left-right)

5-6 (left-right)