Niels Ott - Comments

Kevin Brubeck Unhammer: The USES Issue

nospam@example.com (Kevin Brubeck Unhammer) — Tue, 01 Mar 2011 11:21:45 +0100

Oh dear save us from GUI-enabled academic software! I would much much rather have software that runs on the command-line or has an API, than something that's only accessible through whatever GUI the researcher in question felt was right. I've dealt with enough projects that have prioritised a GUI over command-line/API interfaces to know that it only leads to tears. One project I tried had only a web-interface, which for some tasks was alright, and looked very good, but when you wanted to export any large amount of data for further processing it meant about 20 clicks per export, and any export had to be of a small part of the data... in summary, GUI's do not scale well, and are never flexible enough. (Another project I tried required a you to run emacs in X in order to do text-processing tasks...)

Sowmya: The USES Issue

nospam@example.com (Sowmya) — Tue, 01 Mar 2011 05:50:10 +0100

A course in software design, development practices and IP should be made compulsory to all those who write /USES/ perhaps. Also, I agree on the time/energy factor in making a GUI etc. Apart from that, I think there is also the issue of focus. Perhaps, the "focus" in the research community is to validate if a certain approach works. If it works, it works. Researchers being what they are, might find other priorities rather than making their code standardized, documented and GUI-enabled. May be, there should always be a developer associated with a researcher

Sowmya: Explaining Linguistics with Physics

nospam@example.com (Sowmya) — Tue, 01 Mar 2011 05:43:14 +0100

This explanation makes sense to me. I guess I'd tell this to my mother, who never was a science or linguistics student!

ke: Named Entity list from German Wikipedia

nospam@example.com (ke) — Wed, 03 Feb 2010 14:20:00 +0100

Cool! Your colleague might also be interested in HeiNER: http://heiner.cl.uni-heidelberg.de/

ke: Ärger mit dem Örgele

nospam@example.com (ke) — Wed, 27 Jan 2010 02:21:57 +0100

Neuer Versuch: D.h. im Schwäbischen gibt's einen Unterschied zwischen kurzem e (wie in Bett) und kurzem e (wie in Säcke) und dieser Unterschied lässt sich so charakterisieren, dass Letzteres gespannter ist als Ersteres?

ke: Ärger mit dem Örgele

nospam@example.com (ke) — Wed, 27 Jan 2010 01:43:02 +0100

D.h. im Schwäbischen gibt's einen Unterschied zwischen kurzem (wie in Bett) und kurzem (wie in Säcke) und dieser Unterschied lässt sich so charakterisieren, dass Letzteres gespannter ist als Ersteres?

Niels Ott: Ärger mit dem Örgele

nospam@example.com (Niels Ott) — Wed, 27 Jan 2010 01:28:36 +0100

Das Ö ist in der Tat kurz, keinen Unterschied gäbe es damit aber dann nur, wenn kurze Vokale im Schwäbischen analog zum Hochdeutschen ungespannt wären. Hiller (s.o.) geht davon aus, dass im Stuttgarter Schwäbischen kurze Vokale gespannt sind. Selbst wenn man dem nicht zustimmen möchte, so ist das "Ä" in Örgele und in meinen Ohren eben deutlich mehr ein Ä als ein ungespanntes E. (Leute von nördlich des Weißwurstäquators müssen aufpassen, ein Ä meint hier ein Ä und nicht ein E wie in norddeutschem Kehse, pardon, Käse.)

ke: Ärger mit dem Örgele

nospam@example.com (ke) — Tue, 26 Jan 2010 21:49:28 +0100

Ist das Ö im Örgele nicht kurz, sodass es zwischen Ärgele und Ergele eh keinen Unterschied gäbe?

ke: Simple Readability Formulas And Boring Preprocessing

nospam@example.com (ke) — Sat, 17 Oct 2009 15:20:21 +0200

So you want an estimate of the number of syllables in a word based on the number of letters in the word, right? I don't know if any such data is available, but it should be easy to produce if you have two things for the respective language: a syllable splitter and a moderately large corpus of text: run the splitter on the corpus once, plot the number of syllables against the number of characters, and do a regression analysis. This should give you a nice and, I guess, very accurate function.

al: Simple Readability Formulas And Boring Preprocessing

nospam@example.com (al) — Thu, 15 Oct 2009 20:43:19 +0200

Hi, i've got one question: Are there any formulas or statistic values for getting the average number of syllables for the different languages? It takes a lot of time for splitting the text into syllables. I want to calculate the needed values in real time while the author is writing his text. If there is an average number of syllables per word (depending on the length of the word), i can use these values to get an approximate value. These values may depend on the length of the text too, but i think that the resulting values may be not that bad!?!

Niels Ott: The USES Issue

nospam@example.com (Niels Ott) — Tue, 06 Oct 2009 14:51:19 +0200

As Ted Pedersen points out, the publication of source code must be planned beforehand. So using proprietary software in a project means that one did not have the publication of the code in mind right from the start. It is of course true that the endeavor of releasing your code will face serious trouble in case you cannot do without components with incompatible licensing. But then again, if everybody would think like that, the world would never ever change. If you don't try, you can only fail. (Please use the preview function next time, your formatting turned out to be kind of… confusing.)

Yannick V.: The USES Issue

nospam@example.com (Yannick V.) — Mon, 28 Sep 2009 23:12:30 +0200

Actually, the stuff that I did for my thesis for a long time ran on a single computer used a bunch of data sources with mutually incompatible license requirements was a monolithic blob of modules relied on some C/C++ code that I just compiled manually (i.e., no makefile - someone who doesn't know the code would have to guess which programs to run at which point). Now - I've gotten much better and my parser (which shares some of the code with the anaphora resolution stuff) now uses Python's distutils, but still gcc 4.2 miscompiles part of it and I haven't found out why other people weren't successful in installing it, even though it's a one-line install in the case where it works it still relies on proprietary data sets (notably, SMOR, and a word clustering derived from a corpus that is only for internal use at another Uni) Now, people will say, that's because of the horrid mix of Python and C++ that you're using, and doing stuff for German with pure open source is doomed anyways. So, let's come to the last point, a 40KLoC project for coreference resolution (BART - see www.bart-coref.org). We're working on it with 3-4 people at the same time, sometimes doing different, independent research. But: a lot of time goes into refactoring the system so it doesn't degenerate into a messy blob (and that's with multiple people in there who actually know software engineering) since we're writing and rewriting different parts of the system, it occurs that some modification that's useful for Italian makes the performance for English worse, or breaks things, or makes it not work with JDK 1.5, or activates the hidden function to shoot deadly microwaves at your neighbour. So, in sum it's a lot of work not everyone can do it it doesn't get rewarded at all (I really mean that. And if you ever wondered if it helped to explicitly fund that sort of things - guess again, what will happen is that those people who don't really care about re-use will write nice-sounding proposals and then spend their time creating overengineered ISO standards that no one will ever implement or find useful. CES or LAF, anyone?) * surprisingly often, the approach is just doomed because you need proprietary component X anyway which you can't redistributed

ke: Hohenheimer Verständlichkeitsindex

nospam@example.com (ke) — Sun, 27 Sep 2009 00:39:09 +0200

Having looked at the chart, I'd like to think that the ability to write readable manifestos correlates inversely with ideological stubbornness.

Niels Ott: Text Difficulty and Information Retrieval

nospam@example.com (Niels Ott) — Fri, 28 Aug 2009 15:04:37 +0200

Filling out a sample of their own writing seems appealing, but actually it might turn out to be problematic: after all, this judges their written production, not their perceptive abilities. Concerning words, language learners have a way larger passive vocabulary than an active one. Such a kind of analysis would therefore need an intermediate step to translate from active to passive, which would be subject to research. Apart from that, you're right when you say that users might not want to do this much of work in order to use a search engine. By watching users' clicks, one will always have the mixture of contents and readability: did the user switch to another site because he or she didn't find the text informative enough, or because it was too hard or too easy to read? (Which also brings up the always-present issue of the inter-relation between contents and readability, which would be subject to other research.)

Jason Adams: Text Difficulty and Information Retrieval

nospam@example.com (Jason Adams) — Fri, 28 Aug 2009 14:44:39 +0200

Interesting stuff! So, with respect to users gauging their own quality, I'm guessing that's something you will eventually be able to begin to infer from their clickthrough behavior. I suppose they could also fill out a profile with a sample of their writing you could classify, but that's probably too much work to expect of people.. I wonder how well query classification would work for this purpose. I'm guessing people would make a lot of KWiK style queries that might be useful in determining their proficiency..

Robert Daland: Explaining Linguistics with Physics

nospam@example.com (Robert Daland) — Mon, 23 Mar 2009 16:26:47 +0100

I am in the final phases of my dissertation and as such have been forced to find simple explanations of my field so that I can answer my family when they ask "What do you do?". This fact, that my family doesn't know what I do, is a kind of evidence for the critical role we all need to play in explaining our field to the general public. After all, no one would ask a physicist, "So what is physics, anyway?" Similarly for a psychologist. This is in spite of the fact that a non-scientist has an equally vague idea of what physicists and psychologists work on as what linguists and computational linguists work on. I start off with the simplest possible explanation, give a couple of phenomenon that have received some attention, and then try to end with some applications. Here is a sample spiel: "Linguistics is the scientific study of language. There are a lot of things that we don't understand about language, so I'll just give you an example. You know how when you listen to someone speaking in an unfamiliar language, it all kinda sounds like a blur, but when you listen to a familiar language, you hear it as a discrete sequence of words? There has to be something in your mind that helps you segment the speech stream into word-size units. You might think that it is because you recognize the words, but actually, it turns out that infants can segment speech before they have learned very many words at all. So one question that linguistics tries to answer is, how do infants do that? Linguistics is partly a basic science, but there are lots of applications. For example, Google is trying to improve its search interface. One way they have done this is to detect when someone is asking a question, and try to answer it. But it takes a lot of work to get a computer to understand the meaning of a question. For example, consider what has to happen for the computer to answer these questions: (1) What do skunks like to eat? (2) What likes to eat skunks? These questions both contain almost the same words, but the answers are completely different. That is because the question word (what) refers to the object in the first sentence, but the subject in the second. To answer these questions properly, Google needs to determine the grammatical role of the question word, which requires a pretty sophisticated understanding of English questions. Another example of computational linguistics is machine translation, like when you're trying to read a website in Spanish, you can get a partial translation into English. So this is the kind of thing that we do in linguistics and computational linguistics."

k: The USES Issue

nospam@example.com (k) — Sun, 22 Feb 2009 18:30:48 +0100

"NullPointerException in WeirdClass, corrupting your data" -- haha, I recognize that. I remember really wishing http://casper.sf.net would get off the ground, but it seems to be stuck ----- I once tried implementing two parsers that were described in those 10-page articles for a college project, it took us about 3 months longer than expected due to all the details we had to fill in on our own (or make up, in cases where we couldn't analyze or calculate our way to the correct formulas). In the end, 3 months overdue, we got one of the parsers working at about 3/4 the accuracy that they reported. I still don't really know why it didn't perform as well, since so many details might be different. (We released our own source though , along with the implementational details)

Torsten: CL Blogs and a New Name

nospam@example.com (Torsten) — Mon, 16 Feb 2009 16:24:25 +0100

But can you calculate in your head and talk (meaningfully) at the same time? See, that's why computers will never talk.

Jason Adams: CL Blogs and a New Name

nospam@example.com (Jason Adams) — Mon, 16 Feb 2009 16:17:12 +0100

Yeah, I like the new name a lot better. Updated my list to reflect it.

Christian Pietsch: The USES Issue

nospam@example.com (Christian Pietsch) — Thu, 05 Feb 2009 17:22:07 +0100

Hi Niels, yours is a worthy rant I can subscribe to. I could add a lot of examples from my own experience; mostly obscure software. However, your rant is not exactly original. You have already added the link to Ted Pedersen's very entertaining and illuminating article which has a wider scope, so it is forgiveable that he does not cite previous work in software engineering that addresses exactly the issue you raise here. This body of research is easier to locate if you use the established terminology: what you call USES is either scientific software or research prototypes -- and I suspect that the problems you describe often arise because research prototypes are used as scientific software although they should not be considered a finished product. For literature on these issues, I would take a look at the “Empirical Studies of Software Development” research group at the Open University, UK, headed by Helen Sharp: http://www.springerlink.com/content/w214725153770u22/ http://portal.acm.org/citation.cfm?id=1082983.1083117