Category: Technology

Tuesday’s Results Don’t Matter*

donkey and elephant with voting hatsObviously there is a lot of uncertainty in the polling numbers coming out of the election. Most articles on can be reduced to “we’re pretty sure what’s going on, but also absolutely anything could happen.” Regardless, if we take even the worst numbers for Trump, he’s somewhere around 40% of the popular vote. Taking the 2012 voter turnout as our benchmark (which may be optimistic), 40% of 129,000,000 is 51,600,000. This means there are at least 50 MILLION PEOPLE in our country who are so scared for their jobs, their culture and their identity that they would rather see a Trump presidency than any alternative.

And on Wednesday morning, no matter who wins, these 50 million people will still feel the same. They will still be scared that their way of life is slowly disappearing, and that any (real or imagined) former sense of security is gone. While I  disagree with that notion, as well as the larger sentiment, we also must acknowledge that this is a very real contingent of the US population.

When I go to lectures, classes and meetings, I use real-time captioning to understand what’s being said. Sometimes I joke with my captioners about how my Natural Language Processing classes are teaching me how to automate their job, replacing manual captioning with automatic speech understanding. We laugh and move on, but really this is a microcosm of a larger reality: Technology is steadily and unflinchingly mowing down the forest that is many long-standing, safe and high-skill occupations. The people I work with and the people in Silicon Valley are giddy about all of this.

In William Hertling’s amazing series of novels about AI, the final book takes us to 2043, where humans and AI live in a precarious balance with AI doing most jobs, and most people free to do what they’d like, be it taking classes, painting or traveling. But the people still rise up in revolt against the machines, because they can’t stand a life without purpose. (Interestingly, the Dalai Lama expressed a very similar sentiment in the New York Times 2 days ago.)

So what’s the takeaway?

I don’t think we should slow down the pace of innovation, or compromise our progress in some way. Further, I don’t think the takeaway is some cliched parable about how no development has only an upside, without leaving people behind.

Rather, I think that this presents a unique opportunity for those working in tech, Silicon Valley, NLP, AI, etc. The 50 million people voting for Trump are a sign of a huge untapped market. They are a sign that technological benefits have only been targeting a specific segment of our society.

On November 9th, we need to decide how we’re going to interact with the 50 million people who voted for Trump, regardless of the outcome. Should we be kind and understanding? Sure, because it’s the right thing to do. But at the same time, let’s also acknowledge our own blindspots that they are forcefully drawing attention to, and work to reduce these. I don’t see how we can move forward as a society without both.

* The results do matter; they matter quite a bit. If you’d still like to help make a difference, help fund rides to the polls or send out some text messages. You can still make a real impact.  

Comparing Captioned Phones: ClearCaptions Vs. CapTel

The interwebs are severely lacking any objective comparisons of the two major captioned (landline) telephones on the market: The ClearCaptions Ensemble and the CapTel 840i.

I’ve been using the CapTel for a few years, but the ClearCaptions Ensemble has a 90-day trial period. So I figured I had nothing to lose by trying it out.

Before I get to the captioning quality, which is admittedly the most important aspect, here are a few notes on other aspects of the phones:

Appearance/User Interface
CapTel 840i

CapTel 840i

The CapTel phone is decidedly unsexy. It is pretty old, large and clumsy. I’ve been using the 840i, which does not have a touch screen, for a few years. However, when I went to the CapTel site, I see they now have a touchscreen version, the 2400i. I may have to try that in the future.

The ClearCaptions Ensemble looks much nicer. It is also all touchscreen, except for the power button. However, the touchscreen interface is horrendous. As I said to a friend, “It has a touch interface, but you wish that it didn’t.” When dialing a number, there is no delete if you make a mistake. In addition, the dialpad changes to an awkward double row of numbers when you’ve already entered a few of the numbers you’re trying to call. In short, there is zero usability advantage in the fact that the Ensemble has a touchscreen, and usually it is less usable than the clunky CapTel 840i.

ClearCaptions Ensemble

ClearCaptions Ensemble

Captioning Quality

Obviously this is the most critical aspect of a captioned phone. Below I’ve posted a video with a side-by-side comparison. I used a YouTube video of a person speaking, to ensure that the audio was identical for each trial. Go ahead and check out the video first. I apologize in advance for some of the shaky camera work. My hands were starting to get very tired (see below for an explanation).

As you can see, the speed and accuracy of the CapTel phone is superior to ClearCaptions. Not seen here is the dozen or so trials I did with the ClearCaptions phone, using a different, lower quality video that better portrayed a one-sided phone call. Most of the time, the ClearCaptions phone did not caption anything, and I had to start the call again. The CapTel phone never had any issues with the other video. (This is why my hands (and I) were getting so tired/shaky.)

Additionally, one of the aspects of the ClearCaptions phone that I was excited about is that it supposedly integrates human captioning with automatic/computer-generated captioning. This supposedly makes it faster.As a computational linguist/NLPer, this sounded great! However, as can be seen above, there is no speed or accuracy advantage. When making real calls with the ClearCaptions phone, there are many times when the automatic captions are completely incomprehensible.


While I love sleekness and gadgetry in my smartphone, the most important aspect of a captioned landline phone is reliability: It just has to work. The CapTel phone works faster and more consistently. That’s really all I need to know.

Captioning Around the Country: CART vs C-Print

In the past 6 weeks, I have interviewed or attended Open Houses at 8 different schools around the country. Don’t get me wrong, I am flattered and humbled by the positive responses I received from my PhD applications.

But: It. Was. Exhausting.

Nonetheless, it provided an opportunity to try out different captioning systems and see what captioning is like in places that are not New York City.

First off, at every school I visited, I was able to secure captioning accommodations. It’s a good lesson that as long as you’re proactive and explain exactly what you need, most schools are able to comply. Thank you to all of the administrators and coordinators who helped set this up.

That being said, all captioning is not created equal. The experience made me realize that I’ve been pretty spoiled in New York City, with a relative abundance of well-qualified captionists at my disposal. The following bullet points are largely to serve as a comparison of CART captioning and C-Print, because after extensive googling I found zero qualitative comparisons.

  • The first observation is not a comparison. Rather, it is a direct experience with the phenomena of “discount captioners,” as described by Mirabai Knight, one of the most proficient and strongly activist captionists I’ve used. So-called “CART firms” will troll court reporting schools for mid-level students and use them to charge cash-strapped schools extremely low rates. The result is a terrible experience for students, and a blemish on the reputation of CART captioning.
    • At one school, I actually pulled a professor aside as we were changing rooms and said, “I’m going to have to rely 100% on reading your lips, because I have literally no idea what the captioner is writing.” As Mirabai’s article explains, this is unfortunately all too common, as many schools do not realize that only highly-proficient, highly-trained captioners can provide a sufficient experience for deaf and hard-of-hearing students.
  • CART vs C-Print
    • Mirabai provides a bunch of great reasons why C-Print can fall short of CART captioning. I only used C-Print twice, whereas I’ve been using CART multiple times a week for the better part of 3 years. I’d strongly encourage anyone interested to check out Mirabai’s article.
      • Overall, C-Print was…fine. But when it comes to hearing, “fine” ≠ “adequate.”
      • C-Print does not advertise itself as a literal, word-for-word transcription. Rather, they only “ensure” that the most important things are transcribed. But “importance” is completely at the discretion of the captioner. There were a few occasions where I know the C-Print captioner did not transcribe words that I would consider important, such as the name of an institution where a researcher was located.
      • A C-Print captionist uses a QWERTY keyboard, and depends on a program where they type many abbreviations that the program expands to full words. This usually works well enough, but C-Print is definitely at least 1-2 seconds slower than CART. While 1-2 seconds may not sound like a long time, I would defy you to try having a conversation with someone where things lag 1-2 seconds behind. You’ll quickly see just how significant 1-2 seconds can be.
      • C-Print can be advantageous in noisy situations where an in-person captioner is not available. I used C-Print at a lunch, in an environment that definitely could not have used remote captioning. In this case, a slower, more summarizing transcription is better than a word-for-word transcription that cannot eliminate a high level of background noise.

tl;dr: C-Print captioning is an okay substitution when in-person captioning is not available. But in no way should an institution feel that providing C-Print captioning is the equivalent of providing the transcription provided by CART captioning.

A Funny Thing Happens When Typing Polysyllabic Words

Over the holidays I’ve been emptying my digital pockets and finding all sorts of fun nick-nacks. In particular, Jessie Daniels describes how to be a scholar now, when peer-reviewed articles can begin as Tweets and blog posts. Taking up her clarion call, I thought I’d give it a shot.

[Warning: These findings are minimal and preliminary. A much more thorough analysis needs to be done, and many many more statistical tests need to be run.]

I’ve been meaning to study variation in language production, specifically on a word-by-word basis. For example, how does one typist or one population of typists produce the word giraffe versus another typist or population of typists?

I took a few polysyllabic words from the word list used by Molly Lewis & Michael C. Frank in their recent paper The length of words reflects their conceptual complexityand measured the pause times (intervals) before each letter. Here are the results:


The first thing to note is that pauses before a word are much longer than pauses within a word. This finding is well-established, though.

More interesting (to me, at least) is what happens at syllable boundaries. In the two compound words because and someone, the pause at the syllable boundary is more pronounced. An unpaired t-test shows a significant difference in pause times between syllable-liminal and syllable-internal pause times (< 0.01), whereas differences between other syllable-internal pauses are not significant.

In typing research, a more pronounced pause time indicates “more cognition” is happening. There is some process, such as downloading a word into the lexical buffer, that causes a slowdown in figuring out which key to strike next. It is possible that we are observing a phenomenon where lexical retrieval occurs at the syllable level when a word is made up of multiple words, even if those words do not “compose” the compound word.

Specifically, the word someone can reasonably be decomposed into someone. It might make sense that someone is downloaded syllable-by-syllable, and we see that delay in typing as the next word/syllable is retrieved.

More surprisingly, though, we do not think of because as being composed of because, even though these are two perfectly good words. Nonetheless, we see _something_ happening when the next word/syllable is retrieved.

None of these delays, though, are observed in the words people and about, although I supposed aboutbout.

tl;dr: Something fun is going on with multisyllabic, compound words. It needs a lot more investigation, and I plan on doing just that over the holidays.


Programmers and Pens

To be an outstanding novelist, a writer need not be intimately familiar with the ink-making or printing-press processes. This, however, is the exact state of computer programming today.

Although I only really use high-level languages like Java and Python on a day-to-day basis, I’m constantly employing the more fundamental knowledge I picked up when learning lower-level, more primitive languages, like C++ or Unix. Although knowledge of the latter languages is not “necessary” for programming in the former languages, familiarity with memory allocation and pointers allows me to write more efficient, more robust programs. And more robust programs allow me to test more hypotheses, and do better research.

In the abbots of Medieval monasteries, monks were intimately familiar with how to grind up berries for ink, and stretch out animal hides for parchment. This was necessary knowledge to be a “writer.” Even into the mid 20th century, writers had to know how a pen holds its ink, constantly refilling it, and preventing blotting.

In Daniel Lemire’s most recent blog post, he espouses the view that learning programming is not for everyone, since it’s hard. To be a good programmer requires a lot of work, and being comfortable with delving into technical minutiae.

I love Daniel’s blog, and have previous commented on his thoughts. I think his most recent post, though, shows exactly where we are in the progression of programming, and the dangers that lurk in the immediate future: Programmers are still Benedictine monks, cloistered in hilltop abbeys, showering gifts and knowledge unto the unlearned masses.

Perhaps an argument could be made that the Dark Ages were the result of just this type of schism in knowledge, with one class being scholarly and imbued in the transmission of ideas, while the peasant class was forced to do manual labor and till the fields. There is no reason to assume that a similar division of classes is not imminent, between programmers and non-programmers.

If we train more computer programmers, then we don’t need to worry about a future in which most jobs have been replaced by robots, and the majority of the population is on welfare, or doing mindless tasks. Concerns like these take a myopic view of the future, in which the occupations that currently exist will be the jobs that always exist. 100 years ago, we could not imagine half the jobs that exist today, like “Radiology Technician,” or “Software QA Engineer.” There is no reason to suppose that we haven’t even fathomed 1% of the jobs that will exist 100 years in the future.

However, we need to radically shift our point of view, and concentrate on the accessibility of programming, rather than its exclusiveness, in order to avoid this. There is no shortage of elitism in the programming world. But if we allow this elitism to run rampant, we could be looking at a future like this:

At least the singing and dancing will be good, though…


Our Inverse Relationship With Wifi

A couple months ago, I couldn’t catch the series finale of How I Met Your Mother the night it aired. I didn’t want any spoilers, so I challenged myself to stay off of any news/social media sites the next day, until I could watch the finale.

Needless to say, my self-imposed ban led to a dramatic increase in productivity. I spent more time doing work, and felt more focused when I was engaged in that work. A few weeks ago, I had a similar experience when working from a coffee shop with crappy wifi. By not being able to easily log onto site like reddit or even “helpful” sites like StackOverflow, I was forced to be more attentive to the task at hand. It inspired me to make this chart:


Ubiquitous wifi is a blessing and a curse. It’s great to be able to access anything from anywhere but I don’t think our brains are wired for that. In my linguistics class, Introduction to Sentence Processing, we learned about how “lazy” our mental sentence processor is, and how, as soon as it can form a phrase, it does (rather than considering all hypothetical syntactic options). This leads to errors such as:

The horse raced past the barn fell.

This is called a garden-path sentence, and has been studied extensively. The uptake, I believe, is that our minds are naturally averse to hard work. If a detour, distraction, or vacation is readily available, our instinctual minds (id) will take it, quicker than our disciplined mind (ego/super-ego) can overrule it.

I’m sure there are 100s of productivity “hacks” meant to overcome this obstacle. But, for me, just being cognizant of it is the most powerful “hack.”

Now, I just need to publish this, and post it on facebook, twitter, reddit…crap…