Monday, May 29, 2006

Audio blog, family portrait, and interpersonal whatnot

Blogger offers the option to post audio, but why do you have to call in the audio from a phone? I've got sound files I'd post but fuck the phone shit. Is this some kind of piracy thing? Sheeesh.

Ok to answer the comments questions:

1) The unfortunate family portrait incident
My father's side of the family is Irish and Italian but predominantly Italian in culture. Although there are six siblings (my dad plus 5) only two of them had kids, my dad and his brother Louie. Louie's kids are all girls, and my dad had two daughters and a boy. This meant that for most of my childhood, my sister and I were referred to as "the girls" at extended family gatherings, while our brother was lauded and hailed, given attention, money, and presents for simply existing. This continued through highschool (which my sister dropped out of) and college (which I dropped out of). By 1998, I got married, graduated, bought a house, and started a PhD program. Around the same time period, my brother came out, dropped out of law school, and acquired HIV. In 2000, my family had an extended family gathering. All six of the siblings were there, plus some cousins and second cousins and shit I hardly even knew existed. My dad, my sister, my husband, and I all attended. My brother did not. At this event, my aunt Claire (the oldest of the six) had hired a photographer who would take a big picture of all of us at exactly 6:00. She spent the whole day running around saying things like "the whole family is here!" and "Everyone is here! Oh it's so nice!" She and the rest of the relatives also spent the whole day sucking up to my doctor husband and asking me about grad school and my new house. NO ONE EVEN MENTIONED MY BROTHER. No one.

Picture time came and I had had a few. "Everyone" was arranged in the family room of Louie's house for the big portrait shot. I think if my aunt hadn't squealed yet again about "the whole family" I might have behaved myself. I might have been able to control that evil impulse. But she did it. "It'll be so nice to have a picture of the WHOLE family!" she said. And I, posed at my doctor husband's side with my hands folded femininely on his right shoulder, adjusted my fingers just so....
snap, click, snap, click, snap.

The pictures were taken. My sister came up to me after and said "Oh my god, you didn't! I can't believe you flipped off the camera!" I responded "yeah but only to one shot". Turns out that one shot was the one where everyone's eyes were open. The only one where everyone's eyes were open. And I RUINED THE ENTIRE FAMILY PORTRAIT OF THE WHOLE (except for our unmentionable relative Tony) FAMILY!!!!!

2) Interpersonal coordination
People imitate one another. We stand in the same postures as others around us, we gesture in the same way, if we are asked to stand in a lab and swing pendulums we swing in time to one another (even if we aren't asked to), and we pick up and reproduce even very small details of one another's speech. We do this to different degrees, mediated perhaps by social factors and personality, but it seems even when there is no apparent social gain we do it, and even when there is a social gain and we are imitating a lot, we often are completely unconscious of doing it (personally I love watching my faculty imitate one another's postures and hand positions during talks). Conscious or not, instances or this are called "interpersonal coordination", "entrainment", "convergence", "mimesis", and "imitation". There are slightly different shades of meaning to each term, but they can be grouped together as describing the same class of phenomenon of Monkey see (or hear), Monkey do.

One of the research projects I work on is using this tendency to imitate in speech as a way to investigate the nature of speech perception and the properties of the speech percept (or mental representation of speech/spoken words and sounds). The idea is, very generally, if you imitate a property, you must have perceived that property. The big deal is that most of the classic, traditional research on speech says that we perceive speech in terms of acoustic properties and that we strip away certain speaker specific elements of speech. By this account, when someone says "cat" or even a nonsense syllable like "ka" we are able to understand it because we can match it up with some abstract acoustic representation in our heads. The nature of the something in our heads is what is up for debate in my field, and while traditional accounts differ, they all tend to agree that the something in your head is 100% acoustic and abstract. You have some kind of acoustic template in there that "sounds like" "cat" or "ka" and when you hear "cat" or "ka" you know you heard it because it matched the acoustic properties of that entry (or of those sounds, i.e. "k" plus "a") in your mind.

This something in your head has to be abstract because we all say "cat" differently. In fact, each time you or I or anyone says "cat" we say it differently, at least from a purely acoustic point of view. But you can understand me saying "cat" if I say it with a cold, a sore throat, if I whisper it, if I have a mouthful of muffin (it's a little harder then but if it's not too much muffin you can understand me). So how the hell do you understand it if you have to match all those possible acoustic manifestations of "cat" with this one uber mental acoustic form of CAT in your head?

The theory I work in says that you perceive "cat" not in acoustic properties but in terms of the gestures the speaker made with his or her mouth, tongue, lips, glottis, etc. These properties vary but not as much as the acoustic ones do. This idea is also not new. Theories of recovering the event that made the signal, or inference, abound in the history of psychology and studies of mind. What is new and different is the idea that we perceive the event directly, and not through inference. We "hear" the acoustic consequence of gestures and we perceive in terms of the gesture that made that event. We do not extrapolate, we do not infer, we do not learn an association, we do not need to strip away the speaker specific acoustic properties. That last bit is important because while we might be said to imitate the "abstract" properties (in that we all learn the language we are raised in), we also imitate the not abstract properties (accent, style, voice quality, etc). We can account for that if we describe speech perception in terms of perception of the gestures because the process of "stripping away" so the acoustic input will match with the stored mental abstract acoustic form is not required.

Some people say this is silly, or that it's like magic. You hear sounds, not tongues, says one researcher.

The cool thing about the imitation stuff is that there are some neurological findings of structures and processes which could underly exactly this kind of silly magical perception. There has been a big fuss about what are called mirror neurons, which show activation when we watch another human engage in an action. This area shows activation when we see someone talking which would make sense because we are seeing a human perform an action. But it also shows activation when we listen to speech, which is interesting since it seems to support the idea that you perceive actions (speech gestures) in motor terms.

I'm doing speech research (no neuroimaging...the closest I got to the MRI was as a subject). I was looking for all this background research on imitation in speech as a way to preface our study on imitation in speech. Problem is, we fucked up the imitation study we did so I can't approach it the way we had planned. My advisor hates when I say this, but it's true. We fucked it up, so there isn't much to say in terms of what we perceive in speech or about what the cognitively relevant properties of the speech percept are. But there is something to say about imitation of actions in general, specifically, there is something to say about when and to what degree we imitate if we must extrapolate the action/gesture from the (acoustic) signal vs. when and to what degree we imitate if we perceive gestures directly. So I turned to the non-speech imitation papers and I found some really great stuff. And I am happy. I only hope my advisor is happy enough that she doesn't kill me for going over yet another deadline.

7 comments:

Gypsy said...

i read about that mirror neurons excitement; it explains why i can't finish the story Guts, by Chuck Palahniuk.
i'm excited about mirror neurons.
I've given the sound thing some thought; as a musician lacking perfect pitch, i've always wondered why some people have it and some don't.
If we are learning actions as well as sounds, it would certainly explain why someone whose tone deaf isn't totally crippled in the world.
oh yeah, and way to be contrary w/ your family photo. hopefully, you're brother will think you rock in direct proportion to those familial elements which think you do not rock.
i assure you, you rock.

Mick & Cathy said...

Your a star, I love your attitude in the portrait story.

PFG said...

BF,
I'm not sure about the learning only from audio, not sure if anyone's tested that I mean. The thing is, you get motor information even without watching another speaker if you yourself try to reproduce the sounds you hear (on tape or wherever). If it ends up right, you have a match (acoustically speaking) and you have the proprioceptive (self perceiving) motor information about the gestures that made the sound (along with the negative information as in "well when I did THAT it sure didn't sound right"). Don't know though, I haven't given the learing (first or second) thing a lot of thought. There's a researcher named Catherine Best who has studied this issue from the theoretical perspective I work in. Here's a link to an abstract for a talk she gave:
http://www.arts.usyd.edu.au/departs/linguistics/ling/seminars.html#best

Those poor kids. English is a bitch for this (and other things I'm told by ESL friends). Because here we are saying "in English, the verb is different depending on when the action it describes happened".
Ok. Fine. So the kids try to apply this. They listen. And they hear fully competent native speakers of English saying perfectly grammatical stuff like:
"I take the bus tomorrow."
"So I look him right in the eye and I say 'Josh, you're a loser!'"
"Maureen doesn't drink milk"

All of those are present tense but NOT a single one of them is about something happening RIGHT NOW, and only once. No wonder they get confused. So you probably get early ESL equivalents like:
"I am taking the bus tomorrow" or "I will take/taking the bus tomorrow"
"I was/am looking him right in the eye and saying..."
"Maureen doesn't drinking milk"

cjblue said...

I love the family portrait story. In spirit, your bro was there, He was the finger.

I am interested in your work, really I am. And I find it fascinating - especially when you explain it in idiot terms. But I have to admit the white noise kicked in somewhere around the brain pic. I had you up to there.

This morning, Ariela asked me something. I mumbled a response, something like "innadrawr." She said what? I said "it's innadrawr." She said "In the drawer?" I said yes. She walked away saying to herself "innadrawr."


pzzrj: "Pizza Day", as said by me when sick with a broken toe. Likely to be repeared by my 5yo kid.

PFG said...

Sorry for the white noise. I didn't want to give a lecture so I stuck to the speech stuff mostly. We can chat about the other shit some time when you come over.
(Come over)

I used to do exactly what Ariela is doing. Actually, I still do sometimes. You'd better be careful. She might grow up wanting to be a linguist. Then what are you gonna do? I think show biz holds more wholesome professional opportunities.

Which makes me think of that old song. "Mamas, don't let your babies grow up to be linguists"

I admit, this may be funny only if you have a toradol hangover.

Anonymous said...

She may yet decide to be a linguist. In the last month or so, she has gone from "Singer" a la Dixie Chicks or Indigo Girls to "teacher" to "rabbi"

How cool would it be if she became a rabbi? My aryan baby, all 6 feet of blond haired blue eyed rabbi. Heh. She declared this one to me at Temple last Friday night. She lacks the steadfastness of her sister though. I figure Ariela's choice of career will change a bazillion times before high school. D, on the other hand, might just end up being a rocket scientist, paleontologist & artist.

What's toradol? And how was the weekend? I'll email you.

For some reason, today "choose an identity" (below) is cracking me up. Who should I be? If I were a superhero, what would my power be?

D said...

hello,
In response to the chatter about ESL kiddies. I tried like hell to learn Japanese while living there: failed. My damaged ego told me this lame, but curious excuse: maybe the Japanese female body language is so far removed from my Italo-Australian cultural posture, so unfamiliar, that I simply couldn't pose as anything Japanese, let alone its tongue (NB Male and female Japanese are not the same spoken tongue). Because second language learning requires lashings of imitation in it's early stages (faking essentially until the new language begins to 'reside' in the brain/mouth/hands etc) I needed to, not consciously, but couldn't 'pretend' to be Japanese (embody Japaneseness). It was as if I needed to 'try on' the language before I could 'buy' it and have it as my own. It wasn't an easy fit.

I like the white noise, by the way. Very good short read - funny stuff too.