Words to Pictures, Pictures to Sound

Recently, the Webcomic, Dr. McNinja, the only doctor who's potentially crazier than Doctor Black Jack (who's wound up on some wild adventures of his own) has been setting up the stages for its final arc, resolving multiple plots, and tying up various loose ends that's been built up (and forgotten) over the course of the many years that's it's been running.  While discussion basically revolved around how painfully screwed the Ninja-doc was, I noticed that the closing words comic looked vaguely familiar to me.  A quick look through the archives confirmed my suspicions - it was a deliberate homage to the first arc, back when the doctor was little more than a collection of memes smushed together into a collective taste that somehow wound up tasting great.

That kind of body language and panel composition rang a familiar bell in my head, even as the text was drastically different, especially near the end.  The reason for others not easily picking up on this as me could be contributed to several factors - they hadn't been breathlessly re-reading the series since its conception to familiarize themselves with the current arc.  With all other forms of media entertainment vying for attention (including other webcomics), chances are they even forgot about how the first comic arc ended.

But I think it's more likely that they didn't have my kind of Synesthesia.

I've remarked upon this before, but it bears repeating.  Synesthesia is the trait where one sense (sight, touch, taste, sound, smell) is interpreted a different way by another sense that normally wouldn't be expected to connect that way, and is something that's unsurprisingly, rather common in people with Autism.  When your body's all out of sync with the rest of the world, you latch on to whatever makes most sense.  And for me, that was Sunday Newspaper comics.  Of all the potentially alienating interests out there, I latched upon one that would be socially acceptable.  Like David B. (of Epileptic fame) who embraced the concept of fear, I had to master humour to understand it.  Not an easy task when dealing with old-school comics aimed at an older audience.

When I read words, it's simply a matter of converting these blocks of texts into the closest visual representative I'm aware of.  However, converting pictures is going to require some further explaining.

What I do is assign a different "sound" to each picture, depending on position, posture, colour, and movement, so that when I come across something similar, I'll have something to reference to.  Not unlike how words can have the same meaning, despite having different fonts.  That amount of variety can yield unlimited combinations with limited palettes.  While that doesn't sound like much, consider how many musical notes there are, and how we still haven't run out of new noises yet.  The only difficulty is in trying to accurately describe these "sounds" to an outside audience, since they very rarely come close to how they sound in my head, and I don't have the vocal experience to replicate them accurately.  (Not that they'd have any idea of what I was trying to convey via strange noises alone)

Even though each leaning face has a slightly similar (if different) stance,
I still have an unique "sound" for each, as well as internalized "movement"
despite each image remaining perfectly still.

It's primarily why I've never been able to get into Achewood, Pearls Before Swine, or Dinosaur Comics, because to me, each character design and panel layout are TOO SIMILAR to each other for me to be able to appreciate it in any other way.  Especially with Dinosaur comics.  The strict rigid format makes it almost impossible to enjoy a typical comic, because the setup is the same every single time.  Basically, if you've seen one Dinosaur Comic, you've seen them all, even though the dialogue (and mousover text) is completely different in every one.  That kind of repetition may be fine for people who expect a certain amount of consistency from their enjoyment, but it's a death knell for visual narrative for me, because there's no variation.  The format is so familiar to me that I could recognize the format of a parody within the first two panels alone.

EVERY SINGLE COMIC sounds like this to me.

It's also my main problem in reading S-hero comics from the Golden or Silver age.  The strict narrative page dimensions mean that there's a limited amount of space allotted, so that when I flip through the pages, the signal-to-noise ratio becomes overwhelmingly obvious, resulting in a WHOMP-Whoomp-Whoomp-WHOMP pattern that quickly gets overbearing.  When collected in a large format, and read all at once, The musical rising and falling tone makes for a constant repetitive thudding sound that's painful for me to enjoy.

A page from the Star Wars Manga, staring an overconfident Grand Moff Tarkin.

The ability to detect familiar images is not unlike how obsessive fans notice obvious similarities between Greg Land's traced characters.

It may not have been obvious to most readers, but I signaled out the reused drawing
of Tarkin, from the posture of his smirk and fingers alone.

I was amazed to find out that despite heavily depending on close-captioning for understanding TV and Movies, that others might not like them, and find the helpful text at the bottom distracting.  I relied so much on these helpful tools that alternate tastes never occurred to me.  Likewise, a passage in Temple Gradin's Thinking in Pictures brought forth the notion that Temple was amazed to find out that other people didn't think primarily in pictures, like she did.  It reminded me of a shocking statement made by a Morals teacher - that the very act of reading is in itself, unnatural.  That humans were not meant to look at printed words in a zig-zag pattern.  This was extremely heretical to me, since I spent the majority of my life doing Nothing but reading, and couldn't imagine any other way.  Since then, I've learned about Dyslexics, who struggle to get by without being able to read.  Despite their limitations, they've managed to overcome them via other inventive means, and wind up being productive and innovative members of society, all without being able to read more than a few words (and recognizing some heavily overused ones).

Another surprise was finding out that despite the ease of access, there are people out there who are comic illiterates in being unable to understand even the basic comic panel.  They can't take in both words and pictures at the same time, preferring instead to divide them up separately, examine them at their leisure, and then reluctantly figure out how the two connect.  That kind of analysis is too slow for me, where I see both words and pictures simultaneously.  Having closed-captioned TV probably helped.  I had to read the text at a certain speed to be able to understand what was going on.  Not too fast, lest you run out of time, but not too slow, that you'll run the risk of leaving the sentence unfinished.  At times, I'll prefer to Mute the TV , so I won't be distracted by the captions and sounds not syncing up properly, and run the risk of missing a potentially important end sentence quote.

I've also found that my inner monologue is different depending on the TYPE of captioning used.  For instance, the typically white text on black boxes is vastly different from yellow subtitles from foreign films and Anime fansubs.  Both have their distinctively different sounds, depending on how the sentence is bolded, split up or centered.  There's also the scripting for foreign movies, which has a different kind of pacing compared to the typical Hollywood mindset.  Not every line uttered is intended to be a catchphrase, and is meant to sound more naturalistic; meaning lines that aren't intended to be memorable, but everyday typical dialogue.* Not to mention that there are subtleties that captioning capture that subtitles conveniently leave out.  (Such as names and sound effects)  I found out purely by conjecture that there were minor variations between the captioning and subtitles for GoodFellas, where there were minute differences in the dialogue for each.  The captioning would have minor cues and more instances of uttering character's names, whereas the subtitles would focus more on getting to the immediacy and reductive statement of a guy's jib.

An early scene in GoodFellas, with two subtly different sets of captions.

Likewise for when real-time captioning for the news or Videogame scripts, the scrolling motion of appearing text is significantly different from seeing the whole sentence at a time.  It all depends on how much text I see at once.  Which is why I get so impatient when I'm talking to someone, or receiving instructions face-to-face.  I simply have no idea how long the guy I'm talking to is going to be going on for.

There's an anecdote I read somewhere that probably applies to this.  A young girl from a foreign country saw a Romance Comedy for the first time, and when it was over, she said it was the most terrifying experience she ever had in her life.  The whole move was composed of nothing but disembodied talking heads, cut-off feet, and dismembered torsos.  In short, she was completely unaware of the cinematic language of movies.  That kind of confusion can also apply to comics where the page layout and panel composition can be incomprehensible for someone more used to having everything spelled out for them, and not have challenging notions every second page, such as unexpected plot twists and unreliable narrators for humouristic effect.  The weird thing comes where novice comic readers are baffled by something as simple as a Peanuts or Garfield comic, where confusion shouldn't come into play.

Despite appearances to the contrary, this is NOT a randomized Garfield,
but an actual strip that appeared on September 10, 2005.

Part of the problem may be that there's no general overall agreed mandate for "reading" comics.  Some people are able to take in all the elements of a panel without being distracted by potential spoilers just lurking at the corner of their eye, while others just look at the pretty pictures first, and then go back to read the text to find out what the story was all about.

A commentor from the teaching comics link pointed out that Watchmen is a primer for reading comics for the novice reader more used to prose novels, from the very opening page, starting with a zoom-out.  Reading comics may be second nature to many already acquainted with the form, but can be daunting for those not used to combining two separate art forms at once.  As a kid, I learned to read not by listening, but by looking at the pictures first.  Somewhere along the way, I was able to recognize the weird scribbles on the page for the sounds, and by association, the abstract concepts and objects they represented.  But I have no recollection of how I arrived at that conclusion.  This lack of knowledge of bridging the gap bothers me, since I can't give any adequately helpful advice to others where it applied to me.

A side-by-side comparison from a Manga paying obvious tribute, found on a comic forum.
(Source unknown)

The feat of cross-circuiting senses suggests that having senses compensating for each other would have tremendous advantages, but being able to do so may be harder than we think.  There was a lousy romance movie, At First Sight, about a blind man who got surgery to regain his vision, and he didn't like what he saw, because they didn't look like how he'd pictured them in his mind.  By the same token, when I dare to turn my hearing aids on, I'm dismayed at how different everything on TV sounds from how I expect them to sound.  The music doesn't sync up to the movement onscreen (like Fantasia), and the voices aren't as intoned as I'd expect.  If *I* have so much trouble with getting acclimated with something as intuitive as hearing, how could others be expected to translate smells into colours or touch into music, unless they'd have a deep-seated reason for doing so?  Without proper motivation, there's no point in taking such untraveled off-the-beaten paths unless there's some deeper goal in sight.  At least comics have the advantage of telling stories, so the gateway is easier there, save for blind people.

This posing of two Cosplayers of Jessica Rabbit and Slave Leia reminded me of American Gothic,
and rather fittingly, Leia had her arm wrapped around her "property".

*[One of the hardest lessons I had to learn was that "Not everything that comes out of your mouth has to sound important."  For ages, I would get tongue-tied because I was under the illusion that I had to say something significantly profound every time I spoke, and I would oftentimes be floundering, trying to find the perfect word that was just on the tip of my tongue.]