The Core Dump of Thought

The end

2010-05-01T12:17:00.002+02:00

I'm closing this blog until further, since I do not have much time to invest in writing coherent, meaningful posts. Longer, in-depth texts will from now on appear on my homepage at zvrba.net Commenting is disabled since a lot of spam has lately appeared.

Finally!

2009-11-29T18:53:00.003+01:00

On thursday, 2009-11-26, I have finally defended my PhD! I'll be starting on the new job on tuesday...

Unpredictability of execution times

2009-11-12T11:59:00.002+01:00

I have written an article on intrinsic randomness of modern CPUs that you can fetch here. Originally, I tried to publish it in LWN, but since I haven't heard from their editors in over a week, I'm publishing it this way.

A tricky bug

2009-10-17T14:55:00.002+02:00

Today I debugged a piece of code that looked like this:

printf("foobar\n");
if(fork() == 0)
  exit(0);

When the program is run from the terminal, the output is as expected: a single line containing "foobar". However, when the output is sent to a pipe or redirected to a file, two lines appear in the output. (Actually, the problem was a bit more complicated: there were n fork() calls and there were exactly n additional copies of the output). After a bit of digging around and not finding any obvious fault, I asked about it on IRC.

It turned out that stdout is fully buffered when it is attached to a pipe or a file. What happened is that fork() duplicated also the internal I/O buffers, which got flushed after the child process exited, thus producing extra output. I solved the problem by inserting a fflush(NULL); statement before forking, which flushes buffers of all output streams.

Foxit PDF reader -- no go

2009-09-03T22:05:00.002+02:00

Today I tested the Foxit PDF reader, version 3.1, since many people recommend it as a faster and lighter-weight alternative to Acrobat Reader. Well, its performance is disappointing: opening a 3MB PDF with Foxit takes 2-3 seconds before the first page is displayed. Opening the same PDF with Acrobat Reader (version 9.1) displays the document with no delay at all. I attempted to open few other PDFs with Foxit, with the same result: annoying pause before I could view the document, but with Acrobat Reader showing the document immediately.

So long Foxit, Acrobat Reader is still my favorite PDF reader.

Unexpected behavior of std::bitset in visual studio 2008

2009-08-19T20:21:00.004+02:00

I was writing code for generating prime numbers up to 10^9 (one billion) by sieve of Erathosthenes. So I created a class containing a member of type std::bitset<1000000000> and created an instance of the sieve class within the main function (i.e., allocated on the stack).

When I ran the program, it crashed with stack overflow. Since VC in debug mode implements stack overflow checking, the stack trace was not very helpful -- the program crashes within the _chkstk function. But in my simple test program, the only possible cause of stack overflow could be the instance of the sieve class which contains the bitset member. After lowering the sive's limit to 1000, the program worked fine. It was kinda surprising to learn that std::bitset, unlike std::vector, internally uses an array instead of dynamically allocated storage. (I confirmed this conjecture also by reading the source code of in the corresponding header file.)

OODraw align nonsense...

2009-08-02T09:46:00.002+02:00

I've been drawing some diagrams and I attempted to align (center vertically) several objects using one object as the reference object that should not be moved. But OODraw does not offer such feature; what it does instead is to align the selected objects using their bounding box as the reference, which is completely useless. A workaround is to "protect" the size and position of the reference object and then invoke the align command... which are few unnecessary extra clicks here and there. Stupid, stupid, stupid program!

Git's way to the thrashcan

2009-05-17T10:55:00.003+02:00

I tried to use git-svn and git for a while, only to get fed up with it today and going back to just using subversion. The thing that pissed me off is that I did git rm on wrong files. With subversion, if you do svn revert before commit, the old copy of the file gets restored (and, of course, removal will not be scheduled for commit).

With git, git checkout should, in theory, do the same. Googling about the problem, I found a tip that I should use git reset. However, neither did that restore the accidentally deleted files. Finaly, I experimented a bit and found out that git reset --hard does the trick. Before experimenting, I did, of course, make a backup of the whole repository to another place.

Now, I'm going back to subversion, which is much more intuitive to use. My work is too valuable to be stored in version control system with which I must experiment to achieve even the simplest of operatons. Yes, I found the --hard option in the documentation, but its explanation was so opaque that I wasn't sure whether it would achieve what I wanted.

Next time I'll be wanting to use a DVCS, I'll use Mercurial, as before.

The meaningless school...

2009-04-13T20:23:00.002+02:00

Now, midst in the financial crisis, I've gotten interested in how the economy works in general. Like, the stock market, how can perpetual economic growth be sustainable (in the long run it's exponential), etc. I would also like to know about things that are relevant for real life, like loan calculations, nominal and effective interest rates, forms of long-term savings, etc. And I know zilch about these things.

These things should be a part of basic high-school education. But no, we've been (and today's pupils are still being) overloaded with a bunch of meaningless historical and geographical facts that can be looked up in any online or offline encyclopedia. Among other meaningless stuff that is mostly irrelevant for the practical side of living in the modern world.

Currently I'm busy with writing my PhD, but first priorities on my reading list (after having finished the PhD) are introductory books in macroeconomy, capitalism, and game theory.

Win32 threading API anomaly

2009-01-04T08:42:00.002+01:00

I like to split my code into a series of simple helper functions. When developing for multithreaded environment, some of these functions have to be called while a certain mutex is held by the current thread. In POSIX, it's easy to test for a weaker condition, namely that the mutex is locked:

assert(pthread_mutex_trylock(&mtx) == EBUSY)

There is no way to check that the current thread holds the mutex. However, if another thread has locked the mutex, one may hope that pthread_mutex_unlock will return EPERM error code (I don't think that the pthread library is obliged to check for every possible error condition).

Take Win32, and the TryEnterCriticalSection function which returns, according to the documentation: "If the critical section is successfully entered or the current thread already owns the critical section, the return value is nonzero. If another thread already owns the critical section, the return value is zero."

Since I actually expect that the current thread already owns the mutex, the "or" clause in the first sentence makes it impossible to implement even the (weaker) sanity check which is possible under POSIX. Kinda shame.

Visual Studio 2008, take 2

2008-12-26T09:00:00.002+01:00

Recently I've been extensively working with multithreading in C++, with pthread API, so the code is not portable to Windows. Since I want to get some experience with Windows programming and Visual studio (my holiday project :)), I decided to try out Boost.Thread, which is a portable multi-threading library.

So I downloaded Boost 1.37, the bjam executable for NT, read through the boost getting started manual, compiled the necessary libraries in 32-bit and 64-bit mode as well as a small test application. The process has been surprisingly painless -- it took me about one hour from downloading bjam to having a working test application that comes along with Boost.Thread.

Another reason for wanting to have a portable application is that it is painful to develop over a remote connection to unix server. A compromise solution, which I've been practicing lately, is setting up shared folders with Virtualbox, editing the code with XEmacs, and compiling/testing in the virtual machine. This kinda works, but does not make the code portable :)

New CSPIM release

2008-12-05T21:29:00.002+01:00

I have released v1.0 of my CSPIM MIPS simulator. This release includes some previously outstanding issues, and the simulator can now simulate itself.

CMake and assembly sources

2008-11-29T19:33:00.002+01:00

I've been trying to figure out how to "nicely" integrate building assembler sources with CMake. After some digging around, I found a nice recipe recently posted to CMake mailing list:


set_property(SOURCE cputorture.s PROPERTY LANGUAGE C)

This works because the C compiler, at least on UNIX, recognizes .s as assembler source extension and invokes the proper assembler command.

NetBSD 4 dissapointment

2008-11-26T20:04:00.003+01:00

Well, I installed 32-bit NetBSD 4.0.1 in virtualbox. Then I attempted to install subversion through the pkgsrc system, and.. became thoroughly dissapointed. It pulled not only subversion, but also full apache (although it needs only the apache portable runtime, APR), perl 5.8, ruby 1.8, and who knows what else. I got tired of ruby compiling its "RI", whatever that is, and just deleted the VM. I think I'm going for some desktop ubuntu install now. Or maybe Fedora 10.

Acrobat Reader 9 annoying accessibility feature

2008-11-12T22:31:00.006+01:00

The annoying feature being accessibility which turns itself on, and there's no option to turn it off. Sure, there are a bunch of accessibility preferences, but no "disable" option. So every time I opened a document, I got an annoying dialog that the "document is being prepared...".

Solution: find "Accessibility.api" and "ReadOutLoud.api" files in Acrobat's installation directory (under plugins) and rename them to something else. I have created directory named "disabled" and moved the files there. Works like a charm.

R

2008-11-08T21:44:00.003+01:00

Recently, I had to process and plot a significant amount of data. Previously, I used to use gnuplot for that, but I've discovered two shortcomings: 1) gnuplot has limited data manipulation facilities, so I had to do data processing with perl scripts, 2) it has very arcane syntax and it's rather hard to make it do what you want, especially when you have slightly complex requirements (eg. bar charts of several nested data sets).

I have resisted learning R for a long time, because it also uses a rather arcane language with many complex and rather badly documented data structures. However, a week ago, I remembered my last experience with gnuplot, and I gave R a try. I'm glad I did even though it took me some time to get used to its data manipulation capabilities since the learning curve is rather steep. However, once you get used to it, it's so much easier and faster to manipulate data than with custom-made scripts, and much simpler to produce well-looking plots. Visit its homepage and take a look.

Windows Vista and Visual Studio

2008-10-06T19:51:00.005+02:00

I've bought a new laptop, got 64-bit Vista business with it, so I said - heck, let's give it a try. After one month of using Vista, I don't miss linux desktop at all. I have power management that works all of the time (and that locks the screen after wakeup, without any special configuration!), Flash that does not crash at least once a day (crash on linux manifested most often in the flash plugin eating 100% of CPU), sound system and mixing which works out of the box (no need for mysterious sound daemons that will enable multiple sound applications running simultaneously), plus availability of software for gadgets (garmin gps, nokia phone). I have even found a nice and free virtual desktop manager (VirtuaWin). One month after using Vista.. I won't be installing linux on the laptop anytime soon :-)

You have probably heard that User Access Control (UAC) is PITA, and there are a bunch of cookbooks on the net instructing on how to disable it. Coming from UNIX background, I find it rather natural that you have to type in admin password before performing system-critical tasks. Suggesting to people that they turn off UAC is rather irresponsible, and I have not done it. I like being warned when some operation requires elevated privileges. Oh, and I have turned off the sidebar - gadgets there are mostly useless and they just increase the startup time (which is, I must say, rather decent).

I have found replacement for all of the tools that I used under linux, except one -- development tools. And now, my comparison standard is not linux, but Solaris, with Sun's native developmet tools, CMake and dbx. (BTW, Solaris dbx just rocks! gdb is a poor match for it.) I have installed Visual Studio 8 Professional (I have free license through MSDNAA), and it has several shortcomings in comparison with Solaris tools.

First, I've heard many windows developers say that VS debugger is one of the best, but I find it rather mediocre compared to dbx. VS debugger does not recognize C++ overloaded iterators (so you can't persuade it to show you *it where it is some C++ iterator class) and it is a pain to display deeply nested data structure (just print -r in dbx). VS has even its own embedded command-line interface, but nothing indicates that these features are available through it. Sure, VS debugger is nice for single-stepping through the code, but this is not the way I usually debug the code. (What usually happens in my debugging session is that some assertion/check triggers a crash, I inspect the core dump, put a breakpoint near the offending place, and quickly print few variable values.) On the upside, the VS debugger understands STL containers and can display them in a nice form. But I'd go for dbx any day, only if it were available for Windows.

Oh yes, changing command-line arguments through IDE for debugged programs is PITA :-) If I run the program from the shell and it crashes, it takes several seconds to drop into the debugger, which is kinda annoying.

Another complaint is that VS does not seem to come with a decent profiler such as gprof on Solaris. Most people recommend to use Intel VTune, which costs money. Sun has recently released (free!) SPOT tool which produces a nice report of hot-spots over several runs of your program, in the form of cross-linked HTML pages and graphs.

Then there's DTrace which is currently unique to Solaris and some BSD variants (OS X, FreeBSD?).

Lastly, project configuration management is easier with CMake. I have not yet figured out how I can persuade VS to inherit project properties from the master "solution" properties. Given that projects have parent projects, I should probably play with a deeper project hierarchy. (But why can't I set master settings in the "solution"?! Bleh.)

IntelliSense is nice, but.. it doesn't work all of the time (I see rather often a message in the status bar that IntelliSense couldn't find a completion.. ). Though, when it does work, it sometimes saves me a lookup of member name.

In conclusion - Vista is a rather nice environment for everyday work and it is very stable. However, Microsoft's C and C++ development tools are no match to Solaris's tools, at least when it comes to lower-level stuff. (Maybe not so surprising, UNIX is traditionally C and C++ oriented environment, while MS is moving towards .NET). I'm definitely somewhat less efficient in development with VS (though not drastically); I don't know (yet) whether this is because I haven't learned all the features, or because Solaris tools simply are superior and have no match in functionality in VS.

New site

2008-09-21T19:53:00.002+02:00

You might want to literally waste some time on my new website: riddleschannel.org. The purpose of this page is to conduct certain experiments. If you want to help, just spread around the link!

New code release: ext2frag

2008-08-17T15:55:00.002+02:00

A master student of mine has developed a FUSE filesystem as a part of his master thesis. He needed to compare several fragmentation metrics of his own FS with that of EXT2. However, no decent programs for calculating EXT2 fragmentation statistics are available, so I set out to code one since the student was really short on time. You can get the program here.

Free society ; some tips

2008-07-19T11:09:00.003+02:00

I've recently watched Lessig's presentation on free culture. It has been recommended to me as an example of interesting use of slides, but it is also an excellent lecture on how copyright stifles innovation and development. The following is the basis for the whole lecture:

Creativity and innovation always builds on the past.
The past always tries to control the creativity that builds on it.
Free societies enable the future by limiting the past.
Ours is less and less a free society.

There's another noteworthy quote (by JC Watts, a US congressman/senator?): "If you're explaining, you're losing."

Regarding tips: perl's CPAN shell is an excellent tool to install packages and their dependencies. However, it uses active FTP by default which is more and more dysfunctional because of firewalls. Exporting the environment variable FTP_PASSIVE=1 before running will make it use passive FTP.

Few rants...

2008-06-02T18:19:00.002+02:00

First: I'm switching from emacs back to vim. It's nice to have everything integrated into emacs, it's calculator is especially good (reminds of HP48), but I've found emacs nearly impossible to use for C++ coding. Namely, its internal parser gets often confused, and it won't let you type a simple colon (':'). Usually closing the file and opening it again fixed the problem, but not few days ago. Emacs simply refused to allow me to type in ':' (as in 'public:'). That was too much for me, and I switched back to vim (I've anyway used vi emulation in emacs; i find concise vi-keystrokes much more accessible than emacs' endless strings of characters in combination with alt/meta..)

Second: ubuntu sucks! I have it installed on my desktop machine on work, because that's "what everyone else uses" and only ubuntu CD's are available for installation. Today I ran the latest upgrade of 8.04, rebooted the machine and I was greeted with half-functional GNOME. I.e. the gnome-panel was missing and had I not had some folders on desktop that allowed me to get access to the file manager and eventually run an xterm, it would be completely unusable (i.e. none of the keyboard short-cuts worked either). Solution? I have no idea -- I installed xfce4 to get a functional desktop. The good thing is that xfce4 does not display the annoying update notification every now and then :-D Anyway, ubuntu owes me compensation for 1.5 hours of lost work time. Talk about "no-cost" software :-P

To hell with climate change, time to speak up for the fish!

2008-05-14T18:44:00.003+02:00

Most media are writing about the polar ice melting, polar bears being exterminated.. like anyone's going to notice that they're gone. At the same time, something much worse is happening, and it should deserve at least as much media coverage as climate change: unsustainable overfishing in the seas. This article is a very awakening and scary read. Spread the word and make politicians do something! I've never seen a polar bear, nor do I have a particular desier to see it, but i like very much to eat fish.

[PS. I think that disappearance of any species is regrettable, but I certainly prioritize a food source over some exotic animal. It's regrettable that media don't have same opinion.]

Sudoku solvers and cairo graphics

2008-05-11T19:14:00.002+02:00

I have decided to play a bit with constraint programming, so I'm rewriting my logic game solvers as constraint programs. For that I'm using the minion constraint solver to which I interface through python; I wrote a simple parser of minion's textual output. I wanted some pretty-printing, and doing it in purely textual mode has shown to be rather tedious to code. So I turned to the excellent cairo graphics library for drawing solutions. The library is surprisingly simple to use, and also comes with python bindings. Now I'm working on another puzzle, and after cleaning up the code a bit, I'll upload the new package.

Dent game: an example of nonexistant security

2008-04-13T18:32:00.004+02:00

In Norway, there's a commercial campaign (ending 15.4.) for Dent candies; you're invited to play a game (www.dent.no), and the best scores win some prizes (the coolest prize is a free cinema-ticket for two valid for 1 year). Since the prizes are attractive, I was wondering what kind of security they implemented. Surprisingly (or not; Norwegian companies have a bad track record with security issues), the answer is none!

Before submitting my result, together with my data (email address and phone number), I fired up wireshark and found out the following:

They first connect to stats.indexstats.com to report some basic statistics (a cookie with base-64 url, among other stuff, is sent and an empty GIF is returned).

Then a simple POST request is sent with the score and entered data about the user. Everything through ordinary HTTP, and everything in cleartext! Some binary data is returned, but I haven't bothered to figure out what it is.

So it's trivial to record the traffic into a file, change the score, and resubmit the better score by e.g. netcat. (No, I haven't done it myself -- the hardest part seems to be guessing high enough to win, but still plausible score.)

What could have been done differently? It's impossible to secure an application 100% when the user has full control over the computer, but I think that just AES-encrypting the sent data with a key preshared between the client and the server would have risen the hacking bar significantly. You'd have to download the raw flash file, try to find the encryption key inside it, guess the encryption algorithm, etc. If the encryption key is not stored contiguosly in the file, then you'd also have to reverse-engineer the flash code to see what's going on.

In any case, very badly done web application.

Inkscape

2008-04-02T18:27:00.002+02:00

I've been looking for a long time for some visio-like vector drawing program for Linux. Well, I've recently tried Inkscape, and, while still not a Visio replacement, it does its job pretty well even for technical diagrams.

MIPS CPU simulator, CMake

2008-03-22T16:33:00.004+01:00

Today I've published the first version of a MIPS CPU simulator; you can read more about it here. Even though the version number is 0.9, the program is not in any case beta: it is quite stable, CPU torture tests pass in 32-bit and 64-bit environments, and there seems to be no open security holes, though error "reporting" is sometimes unfriendly: the simulator shall call abort() if the simulated program tries to access memory outside of its allocated range.. but I think that's still way better than doing random stuff to random memory. Work on better error handling is the top priority right now.

I'm also using this opportunity to learn about the CMake build system.

IT Underground 2008

2008-03-01T20:21:00.003+01:00

I have again attended the IT Underground conference in Prague, where I gave a short lecture about PKI in practice. Soon I shall also upload the slides from the talk.

MIPS CPU

2008-02-23T08:34:00.002+01:00

Things have been pretty hectic here, so I haven't had much time to post anything. Recently, I've begun a new personal project: writing a simulator for the MIPS I CPU arhcitecture. I chose this particular CPU because it's the simplest architecture for which gcc can generate code. There are many other MIPS CPU simulators out there, but all of them do more than I need, so I decided to write my own as simple as possible. The ELF loader and symbol lookup are finished, and now I'm working on developing test vectors for the simulator (a very boring part).

As for why I'm doing this - the answer will hopefully be clear soon :-)

Java leaks memory

2008-01-27T08:07:00.000+01:00

Now, this is something to laugh at: when first launching Java, Sun boasted about garbage collector and that programmers will never again have to cope with memory-management problems. Fast-forward to 2008. to read Sun's presentation on avoiding memory leaks.

Some regexps

2008-01-10T15:08:00.000+01:00

I have started to develop some CRM114 scripts to classify Usenet articles. Internally, it uses the TRE regexp engine and I needed a regexp splits article into headers and body; the separation happens at the first empty line. So here's the regexp; it seems to work :)


/((?:(?n:.+):*:_nl:)+):*:_nl:(.+)/

Some peculiarities: the :*:_nl: is CRM114 literal for newline, and the ?n: is a TRE flag to tell it that dot (.) should not match the newline character (by default it does, and this makes TRE different from PCRE regexp). The headers will end up in the 1st matching group, and the article body in the 2nd.

OpenSUSE install and some luck

2007-12-19T17:59:00.000+01:00

Last weekend I installed OpenSUSE 10.3. It was a bumpy installation, since I was without any CD I could burn. So I downloaded the ISO image to FAT32 partition, two additional files (kernel + initrd) and started install. I added another (non-free software) CD image to the list of repositories, but the install had failed afted being done with the 1st CD (Because the CD image was on the partition that was about to be added to /etc/fstab, with no formatting or anything destructive. But that was enough to confuse the installer and make it exit with an error.). So I tried to boot into the installed OS and.. the boot failed. So I was only left with a GRUB command line (because the menu.lst file had not been created) and a WinXP partition which I had to figure out how to boot from.

GRUB built-in help is the most useless help ever.

I needed the chainloader command, ok, but that command wants "blocklist or file name". However, the blocklist syntax is nowhere to be found in the GRUB help. I was just about to go to work and burn the installation CD, when I saw another command (blocklist) which, given filename, prints out a list of blocks. I knew the kernel image filename on the root partition, so I finally got to see how the blocklist syntax is supposed to look like (ie. 0+1 = 1 sector starting from 0th). That enabled me to boot into XP, move stuff to unused partition and successfully install SUSE.

HOWEVER... I had a critical password stored on the root partition which got mkfs'd and overwritten several times.. and it was filled up to 60%, like never before. I thought that I had lost the password, but I tried my luck and used dd to make an image of the root partition into a file. Then I used the strings program to dump all printable strings in the partition image and, unbelievably enough, the password was there, not overwritten by the new data! WOW :) I had more luck than brains this time :)

As for SUSE, I'm running KDE now, and.. it's OK. Livable. Not bad. Portage broke down on my old gentoo installation and I finally got sick of it. Time to spend as little time managing my computer as possible.

Weekend in northern Sweden

2007-12-03T21:49:00.000+01:00

I took a prolonged weekend (fri-mon; 30.11-03.12. 2007) in Northern Sweden; more accurately near the town of Piteå, visiting a friend. Actually, his house is on a nearby island that has only recently been connected to the mainlands with a road. The trip went (by plane) Oslo -> Stockholm -> Luleå (Kallax airbase) where he picked me up with a car. During those few days, we also visited some other places: Luleå, Kalix, and Haparanda/Tornio which lies on the border with Finland. (Haparanda is Swedish name, and Tornio is Finnish name.) You can see all of those places on the map. He bought some spirit drinks in Finland (the shop is named "Alko"), and I bought a bunch of Finnish salmiak/lakris candies. It was interesting to see people in Alko pushing FULL shopping carts of various drinks. Apparently, Haparanda has just recently gotten an IKEA shop, which has boosted the local economy. It is the nearest IKEA shop for that part of Sweden and Finland. Otherwise, it's been a very relaxing and interesting weekend, and they even got me to play bowling; the bowling hall was very nice for such a small town.

I was amazed by Swedish roads. Even though we were kinda in the middle of nowhere, the roads were in good shape (no holes or such) and very well cleansed - there was some snow/ice on them, but despite winter conditions it was possible to drive at speeds over 110 km/h (though with "piggdekk" - spiked tires). Norwegian roads are really bad compared to Swedish roads. (And I mean bad - even in Oslo you can find roads with holes at size of a football-ball in diameter.)

On the way back, I tried to take two cans of surstromming, but it was unfortunately taken by the security officer in Luleå. (BTW, he was really nice - told me that I should have checked that in, asked me whether I was coming back soon [he wanted to store it..]. It's rare to meet a nice security guy.. I guess that small towns have their advantages too :-)) BTW, there's also a technical university at Luleå.

Being far up north, near polar circle, I wanted to experience the "perpetual darkness". Well, it's not perpetual, but pretty close. So I have some pictures too. This picture is some kind of meeting place for young people in Piteå (the two arcs); the most interesting thing is that the picture has been taken ca. 15h.

This picture has been taken at the brightest time of day (though, cloudy one). It is a road that approaches the friend's house. It's a representative picture for most of the roads that we've been driving on.

The following picture shows the frozen sea. It's impressive. Later during the winter, the ice gets so thick that the easiest way to the town is to use a snow scooter and just cross the water.

This picture shows the entrance to the IKEA in Haparanda. I have taken it because the leftmost flag is that of the Sami people.

The last picture is mostly for amusement: it's an announcement in local newspapers that, due to maintenance of the electrrcal network, there will be a short power outage. But notice the time of day :)

Netbeans

2007-11-20T16:19:00.001+01:00

Few days ago I started to write some Java code (I'm using the Choco constraint solver) and tried to use the Netbeans 5.5 IDE (and the version 6 should be out soon). I have to say that I was pleasantly surprised. It is simpler to use than Eclipse, and feels faster. Now I have started to use it for C and C++ development and I'm wondering like... WHY haven't I given it a chance earlier :)

IT Underground 2007

2007-11-10T15:54:00.000+01:00

Yesterday late night (or rather, today early morning) I returned from the IT Underground conference where I have been an invited speaker. I gave a tutorial (as it later turned out, rather advanced) on the ELF file format and shared libraries. I also commented on some possible uses of GOT, eg. to plant an invalid address at a GOT entry in order to trace accesses to global variables. You can get the slides here. Otherwise, conference as a conference - not much to tell :) I attended some interesting lectures and met some new people, and experienced some nice surprises :)

I flew with SAS over Copenhagen. On my way to the conference, after having checked in, I saw that I had been transferred to a later flight, scheduled to leave Copenhagen at 18:30, instead of the earlier flight that was shown on the ticket and which should have departed at ca. 13h. The clerk at the SAS's desk explained that the earlier flight was canceled because it was scheduled to use Dash-8 plane type. SAS had a number of incidents with these airplane types within the last month, and has, for safety reasons, grounded all Dash-8 planes. They have announced to sue the Dash-8 producer, because they suffer enourmous money losses due to grounded planes.

Anyway, since I had more than 7 hours of waiting time, I decided to take a walk through Copenhagen's center. On the metro station I asked some Danish girls about which line to take to the center and overheard another girl asking the same (literally: "I just want to get to the center"). Somehow we ended up entering the metro at the same door and I started the conversation by asking (in English): "Do you also have a long time until the next flight?" She said yes, asked me where I was going and I replied to Warszawa. Then she asked me whether I was polish and I replied that I was Croatian, after which she said in serbo-croatian: "Ja sam iz Novog Sada." (translation: "I'm from Novi Sad"); Novi Sad is a 2nd largest city in Serbia. Then we both started to laugh, were talking much about everything [turned out that she graduated in Russian studies at university in Novi Sad and is now learning Polish language], took a walk through the city center, ate a lunch and took a coffee in a cafe afterwards. Then I headed back to the airport, and she stayed a bit longer in the city, since her plane was leaving 2 hours after mine. It was a very nicely spent time, and when I left for the airport, I realized that we never har introduced ourselves.. I never asked her about her name, nor did she ask me about mine... we were too involved in talk about, like, just about everything.

As for referring to the language as "serbo-croatian".. I'm aware that many Croatians will probably object to this name, but I don't give a sh*t. It seems that I and that girl have very similar views on the language; personally I consider "croatian" and "serbian" being two dialects of the same language that are being kept different for political reasons. I understand what she is saying, she understands what I am saying (except for few words of region-specific slang), so why make a fuss about it? I wrote "serbo-croatian" just to make it explicit that her spoken dialect was of the "serbian" flavor. Nevertheless, her dialect was much closer to the lanugage I'm used to hearing in Zagreb from Zagreb old-timers than the dialect of people coming from Hercegovina and that are considered to be speaking, ironically enough, "croatian" language. Bah, crappy politics. In elementary school (back then when Yugoslavia still existed), we were taught that there were no "croatian" or "serbian" language, but only "croato-serbian" or "serbo-croatian". As much as it is politically incorrect to say it (but hey! Croatia is supposed to be a democratic country with freedom of speech), I personally feel that an attempt to fully separate this (single) languages into two separate languages feels.. not wrong, but just a wasted effort.

It would take me too far to try to explain here the small differences in spoken language between the serbian and croatian dialects. Suffice it to say that the differences between spoken "croatian" and "serbian" languages/dialcets are far smaller than regional differences in the spoken norwegian language (eg. between Bergen - Oslo - Tromsø dialects).

How to raise SW patent hell

2007-10-17T18:03:00.000+02:00

Today I got a nice idea on how to show that SW patents are either total crap or an idea that could put every major SW producer in trouble.

So the plan is this: find a good security engineer and a good patent lawyer. Make a patent titled something like "A method for covertly taking control over software". Make sure that the patent text covers buffer overflows. Next step: sue for patent violation every company whose software has at least one unpatched buffer overflow. I'm sure things would get pretty hilarious in the courts.

A critique of C++

2007-10-14T18:33:00.000+02:00

The usenet article with message ID 13h30fshbtk53eb@corp.supernews.com (google groups has not yet picked it up, so I can't provide a direct link) analyzes in detail what is wrong with the current C++ language. Interesting read.

News and 3-d logic solver

2007-09-22T16:16:00.000+02:00

I have set up a news feed so that you can track changes on my web site. Today's news: solver for the 3-D logic game.

DTrace and C++

2007-09-09T19:41:00.000+02:00

Solaris DTrace isn't that C++ friendly. The SDT provider doesn't work with C++ programs. But I have fortunately managed to find a workaround. Not pretty, but it works.

OOXML vs. ODF

2007-09-01T12:37:00.000+02:00

Do I care whether OOXML becomes standardized? I don't. Why - because I think that it's basically a mistake to standardize a format that conflates presentation and content. In that respect, standardizing ODF was also a mistake. So I don't pretty much care if another such format is standardized. After discussing the issue a bit over a coffe with a friend, I said that they should have better standardized LaTeX. It seems that I'm not the only one who thinks so.

Spam, viruses and addressbooks

2007-08-26T07:24:00.000+02:00

How do you know that someone has added your otherwise unpublished email address to their addressbook(s)? Of course, you suddenly start getting spam! I strongly suspect that there exist viruses/trojans whose only task is to steal data from your addressbook and send it away to .. (someone).

Linus on volatile

2007-08-18T07:34:00.001+02:00

I'm referring to this thread on kerneltrap.org, where Linus is cited to be saying:
- in other words, the *only* possible meaning for "volatile" is a purely
single-CPU meaning. And if you only have a single CPU involved in the
process, the "volatile" is by definition pointless (because even
without a volatile, the compiler is required to make the C code appear
consistent as far as a single CPU is concerned).

He's absolutely wrong in his statement here (namely, that volatile is "by definition" pointless for a single CPU). The C99 standard says that any access to volatile object is a side-effect. This does not mean that compiler optimizations are effectively disabled; rather it means that the compiler must generate memory access instruction instead of caching the value in the register. [This is because accesses to volatile objects may produce side-effects, so the read value may change between reads without an intervening store instruction.]

Now, consider the following simple code in a uniprocessor configuration:

while(!flag) ;

which simply waits for the flag to become true (e.g. set by an interrupt handler). If flag is not declared as volatile, the compiler might well generate an infinite loop, but if the flag is declared as volatile, the compiler must generate code that will check the physical memory location in every iteration and thus the flag change from an interrupt handler will be detected.

So, volatile is important even on uniprocessors whenever there is a possibility of executing asynchronous code (e.g. interrupts). And its semantics is defined well enough to prevent errors like the one I have described above.

Oh well, I don't really care nor shall I bother to comment on this on the kernel mailing list. It's his kernel, I use it only on my desktop and I don't really care what future impact this change will have (and it might have far-reaching consequences that are really hard to discover; as any bugs related to asynchronicity). I have said long time ago that I'd never consider Linux for "serious" applications (i.e. anything else than a cheap desktop), and such displays of blatant ignorance by its leading developer just make my stand firmer.

More python brokenness: distutils

2007-08-08T18:43:00.000+02:00

So I tried to compile Mercurial on Solaris with Sun's native compiler. I set CC and CFLAGS environment variables, but, as documented in distutils manual, the CFLAGS set in the environment are added to some predefined set of flags. This set of flags was adjusted to gcc, and Sun's compiler broke (no wonder) on some of gcc's options. It seems that compiler options are hardcoded in distutils at some obscure place. "Great" work!

After ca. 30 mins of fiddling around and searching on google, I recompiled the module using gcc.

Concurrent shared memory allocator

2007-08-06T20:15:00.000+02:00

I have recently adapted Solaris's libmtmalloc(3) to satisfy allocation requests from shared memory segments; it works both with POSIX and SYSV shared memory. You can get the code here. I knew about a similar project, but I have no idea how well does it scale with increasing number of concurrent processes.

Prex, DTrace and linux lameness

2007-08-04T12:15:00.000+02:00

Adam Leventhal has in his blog exposed the lameness of the SystemTap team (e.g. they claim that they weren't inspired by DTrace at all, but output format of some of their utilities is identical to that of DTrace). He also took an amusing photo showing that users are not very happy with SystemTap.

But I want to point to another very useful tool that got completely shadowed by DTrace: TNF tracing. In short, it is "printf() debugging" on steroids; read the tracing(3TNF) man page for more details. The TNF logging probes must be manually inserted in the source code at places where printf() would be put, and they produce no output by default. When you run the program under the control of prex(1) program, you may choose to log the otuput of all probes, or select just a subset of them (they are named!). The trace is later analyzed with the tnfdump(1) program. The best thing is that prex works also on running programs; it can attach to them, collect data and detach at later point.

TNF has been an enormous help in finding and correcting complex, time-dependent problems that I'd have much harder time to resolve otherwise.

Solaris signal handling bug, part 2

2007-08-03T20:31:00.000+02:00

My initial report has been confirmed to be a genuine bug. Even more interesting, the bug has been introduced in Solaris 8, and hasn't been noticed until few days ago. Wow :)

A possible signal handling bug in Solaris

2007-07-29T19:45:00.001+02:00

I know that combining threads and signals is a bad idea, but I just couldn't resist the temptation. It seemed like a nice solution until I've hit what seems to be a possible signal handling bug: sometimes the signal is not delivered on the alternate signal stack, even though a thread has requested so. You can read the full description here. I wonder whether I'll get a reply. In the mean-time, I'm abandoning threads in favor of multiple processes operating over a large shared memory segment.

Summary

2007-07-08T19:40:00.000+02:00

It's a very busy time. I've passed (with the best grade) my final exam in Norwegian, and since then I'm busy with coding my PhD project. Have been busy with installing Solaris on some machines and bringing it into usable state (by installing many small but important packages), upgrading the compilers to latest versions, learning the dbx debugger, fighting with compiler bugs and discovering subtle but important restrictions of lock-free data structures.

Writing code also involves learning much about the C++ boost libraries and discovering some important weaknesses of C++ (this time: operators new and delete are class members which is very inconvenient if you want to impose external allocator onto existing classes). But I was also impressed how good the Sun's C++ compiler is when it comes to aggressive optimizations and heavy inlining of C++ code; this is very important with heavy template metaprogramming.

Each of these topics is interesting enough to deserve its own post, so expect more posts in the upcoming days.

X11 ssh forwarding

2007-06-16T21:37:00.001+02:00

It is common knowledge that ssh -X or ssh -Y forward X11 connections. However, what I didn't know until few days ago is that this works even across several machines. That is, if you do ssh -Y from A -> B -> C, then opening an X program on machine C will forward the X connection to the X server at machine A. Very convenient.

Windows Research Kernel

2007-06-13T19:00:00.000+02:00

This is interesting: Microsoft gives out the NT kernel source for academic, non-profit use. Too bad I don't have much time to play with it a bit; I've always been impressed by the very NT kernel (which is itself VMS-ish). More details here. A related project is ProjectOZ, a platform (simulator) for teaching principles of operating systems.

Window managers

2007-06-10T21:58:00.000+02:00

After being a long-time user of the fvwm2 window manager, I have switched to Ion. The author, incidentally, dislikes the current state of open-source software, linux (calling it the Idiot-box Linux), and gnome. I share many of his opinions and have subscribed to his blog. Very interesting read!

One end and one beginning

2007-05-27T16:18:00.000+02:00

Today I have closed the old web site where I and a friend of mine have tried to build a community. Building a community is hard, takes effort, and consequently, time. And time is a pretty scarce resource for us. We have decided to take down the old web, and I have moved the content to my new personal web: http://zvrba.net

Digitally signing files

2007-05-25T16:18:00.000+02:00

When you want to digitally sign a file, you can either make an embedded signature or a detached signature. The disadvantage of the embedded signature is that the original format is mangled. You have to "unpack" the file to another file (the process of verification does this) to be able to use it. Not very convenient. Detached signature preserves the original format, but stores the signature in a separate file. Unless you have a convention and hold firmly to it, you will have problems with pairing up signatures and their corresponding files. Modern filesystems open up a third possibility: a signature can be placed into a file's extended attribute. This solves both problems: the file's original format is preserved, and the signature is still attached to the file.

Algorithm toolbox

2007-04-29T20:22:00.000+02:00

I have released a preliminary version of an algorithm toolbox. The toolbox includes van Emde-Boas trees, intrusive AVL trees (thus, applicable in scenarios where there are no memory management facilities), and exhaustive permutation tester. The code is written in C++ and may be obtained here.

Solaris 11, xemacs and dbx

2007-04-24T15:50:00.000+02:00

I have recently installed Solaris Express (Solaris 11), first in one virtual machine and then on "bare metal". I have to say that I'm very pleasantly surprised. The installation went very smooth, the only "configuration" neccessary was to enter time, location, select additional locales, type in the root password and to create a new non-root user upon first login. The default desktop is GNOME and looks very polished up. Security-wise, the installation seems very sane. There are many services bound on TCP ports, but almost all of them only to the loopback interface (meaning that only local programs can connect to them). The only publicly open ports are 111 (rpcbind) and 22 (ssh), 6011 and 6010 (I guess, X). And sshd is configured to not permit root logins by default.

I tried Sun Studio 11, but.. being used to emacs, it doesn't feel quite right (+ it's not comfortable to use it if you're running it over remote X11 connection). However, Solaris has a very nice toolbox for programmers: dbx, mdb, dtrace and tnf (man tracing), different malloc libraries, etc. It's really very advanced OS, and very developer-friendly. Plus there are zones, fine-grained privilege and role system which works... And everything is excellently documented in man pages. I'm too short on time right now to describe all the goodies, but.. I'm impressed. By the installation, default settings, and overall consistency and look&feel of the system. I can only say excellent work to all the Solaris developers, and a big thanks to Sun for making it all available at no charge.

Now, to the topic: I had a problem with xemacs not displaying source files when I used it with dbx to debug a multi-directory project. The problem turned out to be that dbx does not by default display full path to the source file. xemacs looks for the file in the wrong directory and consequently does not show the source you're supposed to be debugging. You can solve this problem by putting the following line in your .dbxrc:


DBX_output_short_file_name=off

Linux signal handling is broken

2007-04-10T17:54:00.000+02:00

Enter sigaction() in combination with SA_SIGINFO flag. Such signal handler accepts three arguments, the third being the context (full machine state needed to resume it, eg. registers) of the interrupted thread.

First problem: linux ABI is broken. The FP state in the uc_mcontext member of the ucontext_t structure is pointer, instead of value. This makes copying of the context nontrivial.

Second problem: You can't use setcontext() to leave signal handler and jump into another, previously saved, context. (Or, for that matter, you can't use it to return to the very same context passed as argument to the signal handler.) In other words, signal handler like

static void sighandler(
  int signo, siginfo_t *psi, void *pv)
{
  memcpy(puc_old, pv, sizeof(ucontext_t));
  /* choose another context to dispatch */
  setcontext(puc_another);
}

does not work. It does not restore signal mask specified in the puc_other, does not reestablish alternate signal stack, etc. However, this scheme works flawlessly on Solaris.

How am I fixing it on Linux? I walk the stack frames (following the saved stack frame pointer), modify the return address so that the signal handler returns to itself instead to the interrupted context, etc. Very ugly and nonportable.

Not to mention that I'm relying on luck: it seems that, under current linux kernel, it is not possible to atomically restore signal mask and return from signal handler to context other than the immediately interrupted one. (Heck, it's not even possible to do it nonatomically without resorting to "black magic" involving reading hex dumps of stack frames.)

I'm installing Solaris in a virtual machine to try it out, and I'm seriously considering to move my development to Solaris.

Three jewels

2007-03-28T17:50:00.000+02:00

This one is, I guess, going to be another classic in a few years. Andrei Alexandrescu stated in comp.lang.c++.moderated:
Well, for my money, using iostreams already puts the code in a low quality bracket :o).

The thread was about how people are dogmatic regarding the goto statement, so here are two more links:

D. E. Knuth, "Structured Programming with goto statements" [PDF]
Frank Rubin's response [PDF] to the Dijkstra's paper on "harmful goto", published in readers's letters, Communications of the ACM, March 1987, Volume 30, Number 3.

Subverting the GPL (again!)

2007-03-23T21:51:00.000+01:00

If you use some GPL code in your program and you distribute that program, than you also must make the program source available. A fun thing would be to comply with the GPL, but publish the source code of the parts of the program that you wish to keep "secret" in obfuscated form.

Here's a relatively easy way to do it: put "proprietary" parts of your code into separate source files, compile them to object code and decompile the object code back to "source". "Nice" decompilation is of course impossible. But we don't want nice - we can just generate code full of gotos, primitive machine instructions, and variable "names" which correspond to the variable's address in memory after the program is loaded.

I think that it would work, because the license states only "Accompany it with the complete corresponding machine-readable source code..." Such source code is undeniably machine-readable (and even compileable!), and it seems that distributing a "proprietary" program in such form would comply with the license.

Comments?

Another look on climate change

2007-03-16T16:03:00.000+01:00

"This has been the warmest winter ever", "We must reduce CO2 at all costs", etc. You can hear such things pretty often today. But take another perspective: the global warming, which caused the warmest winter 'ever' (highly doubtful), also saved huge amounts of energy that would otherwise be spent on heating. Less energy used on heating means less generated CO2 this winter. Why doesn't anyone mention that?

Hungarian notation

2007-03-10T20:03:00.000+01:00

You have probably heard that hungarian notation is awful and that it should be avoided. However, in most of the Windows APIs, the notation has been misused. The intention behind Simonyi's notation is akin to dimensional analysis. I recommend you to read this article as well as Simonyi's original paper.

While having a good intention behind it, the idea is actually just a fix for a defficient language, namely C and C++. If typedef were not a mere type alias, but a true new type, this convention would be (almost) unneccessary as the compiler could catch all errors. For example:


typedef int Temperature;
typedef int Volume;

int f(Temperature a, Volume b)
{
  return a+b;
}

Although it is meaningless to add temperature and volume quantities, this is perfectly legal C and C++ code, because Temperature and Volume are mere aliases for int.

C++ templates have strict compile-time type-checking, but the syntax is unfortunately just horrendous.

Giants meet

2007-02-15T15:11:00.000+01:00

I am actually not surprised about this. I'm just wondering why didn't it happen sooner.

A thought on neural networks

2007-02-10T10:43:00.000+01:00

Neural networks are a (currently not very successful) attempt to programmatically mimic the learning behaviour of a human brain. Research in neural networks has mostly focused on topologies and transfer functions in the nodes. However, the aspect of time has been neglected. Human nerves transport signals with (relatively slow) speed between 0.5 and 120 m/s (quoting a random reference from the web).

Could it be that our learning capability depends not only on particular signal values (the part that artificial neural networks are simulating), but also on propagation time between neurons in the brain? A signal might have different effect on a neuron in the brain, depending on the time of signal's arrival.

This would add another dimension to artificial neural networks: temporally changing transfer functions in the nodes. This is just an idea for further research, maybe someone has already looked into it.

x86-64 ISA oddity

2007-02-03T08:37:00.000+01:00

New processors have FXSAVE and FXRSTOR to save/restore the floating point state of both x87 and SSE units. In 64-bit mode, PUSHAD and POPAD instructions generate undefined operation exception, there is no new instruction to save/restore all general-purpose registers, and the hardware task switching is disabled. I'm wondering why did the AMD designers decide to cripple the CPU in such a way (along the side of disabling segmentation, but that's a story for another post).

Enforcing mandatory exercise at work

2007-01-15T17:46:00.000+01:00

I'm working in a large but low (only 4 floors) building with many printers. Today I came upon a slightly evil idea. Instead of letting people choose the printer on which to print their documents, the administrators should arrange the system to choose the destination printer randomly. Benefits would be twofold:

People would be forced to walk longer distances, which is healthy. This is beneficial especially for people working at computers, sitting most of their working hours.

I also guess that paper would be saved: in order to avoid the previous point, people would avoid printing stuff unless they absolutely have to.

To make the first point most effective, people should be locked out of elevators, except disabled people which would get their own "elevator key".

Assembler as a first "language" ?

2007-01-07T20:42:00.000+01:00

I participated recently in a discussion where someone suggested to teach programming to beginners with assembler as the first language. There were far many more negative than positive reactions. My opinion is that assembler is just a tool to show concepts in practice. If used properly, it serves as well as any other tool (ie. high-level programming language). Here are Knuth's reasons for using assembler in his TAOCP books.

Now, I agree that it is much harder to make a good course with assembler - topics, examples and problems must be chosen much more carefully than for a high-level programming language. And x86 is a wrong choice as a first architecture - something like Knuth's MMIX or the MIPS architecture would be much better.

Bottom line is that, while there are admittedly many traps, I don't see anything fundamentally wrong with such approach. Your opinions?

AMD64 TLB invalidation performance

2006-12-24T22:32:00.000+01:00

The AMD64 optimization manual specifies that the latency of INVLPG instruction is 101 cycles in 32-bit and 80 cycles in 64-bit mode. Considering that the TLB is so closely tied to MMU, CPU, and that no accesses to RAM are needed, I'm wondering why is it so slow? Even more interesting is the large (20%) difference between 32- and 64-bit mode.

How fast/slow is it on Intel CPUs? No idea. Their optimization manual gives instruction latencies only for a relatively small subset of instructions. INVLPG is not among them.

Ashgabat

2006-12-23T12:26:00.000+01:00

After hearing the news about the death of Turkmenistan's president, I searched the net to find some pictures of the capital, Ashgabat. I found this site and all I could think of after seeing the pictures was "Wow. I want to go there." (despite that summer temperatures can rise up to 45 degrees Celsius) Colourful and a bit surreal. Jewel found in an ex-Soviet republic. Who would have thought?

Peter Gutmann on Vista content protection

2006-12-23T08:49:00.000+01:00

His article is a very interesting read. A quote: "The Vista Content Protection specification could very well constitute the longest suicide note in history." The scary part is about automatic picture degradation while listening to music and its potentially disastrous consequences in medical applications where crystal-clear picture is needed for correct diagnosis.

How to not debug programs

2006-12-13T14:44:00.000+01:00

I was drawn to this article, titled "Signals as a Linux debugging tool", by a recent link on the OSNews site. The subtitle writes "Intelligent signal handling finds bugs faster". Two things are wrong about this article.

The first thing is that the author's code examples have printf function in a signal handler to output register values when a fault happens. This is undefined behaviour, as the printf function is not listed as async-signal-safe in the POSIX standard. Ironically, an article titled "Use reentrant functions for safer signal handling" is listed among the references.

The second thing is that the author suggests an unbelievably complicated way of finding bugs. Once your signal handler with undefined behaviour has dumped the values of registers to your terminal, you are supposed to use objdump to disassemble your program executable, find the faulting location, and somehow map it to your program source.

I wonder whether the author actually knows how to make a program dump core when it faults, what to do with the core file, and how to use a debugger, such as gdb. (Hint: debuggers are much more powerful and convenient to use than what the author suggests in the article.) It's surprising that such a misleading, low-quality article can show up on an IBM's web site.

Nokia 770

2006-12-06T16:55:00.000+01:00

Recently I had the opportunity to get Nokia 770 Internet Tablet (basically for free - a loan on indefinite time). I declined. Previously I have owned both a PalmOS device (Visor), and an iPaq onto which I installed Linux. In both cases I couldn't think of anything useful I could use the device for. Otherwise it was just a waste of time.

This time I did have an idea - as a dictionary where I can note down new words, and as note-taker to replace my paper notebook. So I tried to play with it and it has the same problem as other PDAs - text entry. The on-screen keyboard you tap on is next to useless for real-time text entry. The handwriting recognition isn't much better. If you want high accuracy, you need to carefully and slowly draw letter by letter. I wonder why Nokia didn't buy patent for the Palm's Graffiti entry system. (Yes, there is an option of external Bluetooth keyboard. But that is too cumbersome - suddenly you have to fumble around with two small pieces of fragile equipment that you don't want to drop.)

An ideal PDA should come with software capable of learning natural handwriting. Here's how I imagine it could work. First you type in (using a keyboard) a relatively large text. Then you write the text (using your natural handwriting) on the PDA and let it figure out for what groups of letters composite pen strokes stand for.

This learning process would initially take more time, but it would be worth it in the long run. I don't think that today you can find a PDA which can in real-time recognize natural handwriting. Until such time comes, I think I will stick with pen and paper for taking notes
in real-time.

C, C++ and width of integer types

2006-11-27T15:56:00.000+01:00

Perpetual problem with C and C++ programming is very loose specification of integer types (eg. int must be at least 16 bits, without upper limit, long has to be at least 32 bits and not smaller than int). C99 solved this problem by introducing the <stdint.h> header.

The problem could have been solved by redefining the register keyword to mean:

I wonder how many programs would break, esp. if the register keyword would be no-op if its use would violate the minimum requirements for integer types (short, int: at least 16 bits, long: at least 32 bits).

I don't how useful would this redefinition be. Afterall, C99 does include "fast" integer types. Hm. Any opinions?

Spammers have gotten smarter

2006-11-25T18:30:00.000+01:00

Today I've actually read some text that got through the spam filter. Total nonsense that kinda makes sense. Here's an excerpt:

Popularity of blogs helped also to popularize concept web content mechanisms is such as or Atom have. Xml am perform operations instead using or Feeditem Item want or cannot changed exception readunread property in attached. System time or downloaded via Http Https parsed is normalized unified Identifies updated is Merges reflect last. Those tricky a details platform shields even of supports upcoming Support or Whether is implement innovative scenarios basically deal with Common Feed List. [etc]

This reminded me of a computer-generated text so I sought for Markov chain text generators. Here's one for example. Study its output (links are near the bottom of the page) and it'll be the same kind of "nonsense making sense".

Bayesian filtering is a kind of "inverse" of Markov chain text generation - both methods are based on statistics. The problem with the Markov-generated text is that its statistical properties closely match those of real text, so the Bayesian filter doesn't classify them as spam.

Generating garbage with required statistical properties is relatively easy; it just requires a list of words and a good Markov model. Once generated, it requires real human understanding for classification.

I didn't study theory behind Markov processes and Bayesian filtering deepely. I might be talking half-rubbish :) But given the amount and kind of spam that gets through the filter, I have a feeling that spammers are slowly winning the battle.

10 Immutable Laws of Security

2006-11-20T21:23:00.001+01:00

This is a nice essay.

New C++

2006-11-20T15:56:00.000+01:00

This article describes the new features of the upcoming C++09 standard. (Read through it, there are other good links hidden inside). My favorite addition is automatic type deduction (unfortunately, not described in the article).

NILFS for Linux

2006-11-15T19:12:00.000+01:00

Today I've seen a reference to NILFS, a log-structured filesystem for Linux. It's interesting how they've put the most important "feature" at the bottom of the "Current status" page. Looking at the page, the first important thing one notices is that the work on garbage collector is ongoing. Doesn't sound well. At the bottom of the page, they conclude under known bugs with "The system hangs on a disk full condition." How nice :)

On a more serious note, I think that it's great that someone is working on alternative filesystems. NILFS can, for example, support "time travel".

XML sucks (again!)

2006-11-08T21:59:00.000+01:00

This isn't yet another of my anti-XML rants. There actually exists a web-site with such name- xmlsucks.org.

2006-11-06T21:42:00.000+01:00

I've registered myself with LinkedIn. It is a service to help maintain professional contacts. Doesn't cost anything and might be useful in the future.

Makefile madness

2006-11-02T18:59:00.000+01:00

Whenever I start a new project, there is one single thing I hate the most: maintaining a makefile. (Relatively) long time ago I found the article "Recursive Make Considered Harmful". It is both a critique of recirsive make, and a guide on how to write a good makefile.

I bit the bullet, applied the recipes given there (it didn't even take much time), and so far so good: it discovers automatically newly added files and dependencies maintain themselves. Without resorting auto*madness :)

Software vs. hardware virtualization

2006-10-30T16:11:00.000+01:00

This article [vmware.com] compares performance of software and hardware virtualization techniques. What is most surprising is that the results are mixed: hardware-assisted virtualization can actually be slower than pure software virtualization under certain workloads. Wow!

The benchmark was done with VT-enabled Pentium 4. It would be interesting to see how the AMD's Pacifica hardware virtualization compares.

A small laugh

2006-10-29T07:43:00.000+01:00

Being a Croatian citizen, I found this news article funny: according to the "Worldwide Press Freedom Index", USA ended up at 53. place , together with Croatia, Botswana and Tonga.

What's even more surprising is that some former communist countries, which only recently abandoned communism are extremely high on the list - eg. Czech Republic is at the 5th place.

Anti-virus, virtualization and security paradigm

2006-10-28T08:05:00.000+02:00

This is a very interesting interview with Joanna Rutkowska, the author of a "Blue Pill" rootkit. She just confirmed an opinion I had for a long time: that AV programs are mostly useless (heck, she doesn't even run one on her WinXP 64-bit machine).

AV detection is an inherently undecidable problem; therefore it will always be possible to create an undetectable virus. Without needing a rootkit that puts the OS into a VM.

Her wish (quote):

"The solution that I would love to have would be based on integrity checking of all the system components, starting from filesystem (digitally signed files), through verifying that all code sections in memory haven't been modified (something I partly implemented in my SVV scanner) and finally checking all the possible "dynamic hooking places" in kernel data sections."

is not realistic (unless the scanner is in the hypervisor) because of the question: How does the scanner ensure its own integrity?

What I would like to see is a paradigm shift in the security industry. It should put more weight on prevention and damage containment rather than source code auditing and scanning of programs/memory. Both techniques have been in use for a very long time and they don't work very well.

My view is that the OS should use the virtualization technology to create extremely light-weight, isolated environments; in the extreme case 1 VM per running application (this requires some heavy engineering to be doable efficiently - eg. sharing of the core OS code between VM instances). Each VM would expose only those parts of OS functionality that is absolutely neccessary for the application to work. Information flow between VMs would be strictly under user control (thus, making the user once more the weakest link in the chain).

There lie some heavy research questions in my proposal:

Efficient memory utilization (it would be infeasible to completely copy all of the underlying OS into each VM). Hypervisor would have to be intimately tied to the "guest" OS.
Policies for information flow between VMs.
Efficient history saving (so that user can roll back to some previous VM state).
Interoperability with other VM products like Xen or VMware.

Regarding the last point, there is an interesting comment in the AMD64 Pacifica manual for the VMRUN instruction under "Instruction intercepts":
"Note: The current implementation requires that the VMRUN intercept always be set in the VMCB."
Is this a hint that, in the future, we might get HW support for recursive virtual machines?

Object-orientation

2006-10-18T20:11:00.000+02:00

I have kind of despised object-oriented programming for a long time now (This was a result of bad experiences on large projects. It actually made things worse than better.). Until I found this section in Xavier Amatriain's PhD thesis. It is a nice view on the matter from Kristen Nygaard, one of the "fathers" of object-oriented programming (the other was Ole-Johan Dahl). His view I actually like.

After reading that section in the thesis and browsing through Nygaard's and Dahl's homepage, I felt a bit sad for not getting to meet them in person. Now I respect for OO as it was envisioned and find it a worthy idea. Misused in practice most of the time.

Tags: programming

Interfaces and stability

2006-10-17T16:21:00.000+02:00

These days there is much fuss around (not so) newly discovered bugs in nVidia drivers for Linux. Instead of being happy that a large software vendor has gone to trouble of providing drivers for a nonsignificant portion of their market share, users are whining about the "evil" nature of closed-source binary blobs being downloaded to kernel.

The importance of the bug is exaggerated. I consider it a bad practice to install any kind of advanced graphic capabilities on servers. As for desktops.. well, a plethora of bugs in other "desktop programs" already exist, this one doesn't make any additional threats beyond the existing ones. And it's simple to fix - don't use the driver.

I've found many complaints that nVidia's drivers are low-quality, unstable or just don't work. (Even today a friend complained to me.)

What is most fascinating is that users of these drivers are barking at the wrong tree (nVidia in this case): the real fault lies on the lack of official kernel APIs which are also ever-changing, to make the situation more difficult. And Linus is even proud of it, replying in the lines of "read the source".

IMHO, users are in this case a direct victim of such attitude. As I said, Linux is only a secondary platform to nVidia. They have no real financial incentive in keeping up with Linux kernel development. There is no point in constantly keeping behind myriad of Linux kernels with different patches and trying to make drivers work with every single one of them. Why? Because there's no stable kernel API.

Binary-only drivers (if written well, but that's beside the point here) work very well on Solaris, AIX and Win32. I don't know about AIX, but I know that Solaris and Win32 publish official driver development kits (DDK). Every 3rd party manufacturer can write a driver w/o relying on the "current state of flux" of the kernel and be reasonably certain that their investment in the platform is long-term. Something which is not the case with Linux.

I encourage users to stop buying the "binary blobs are evil"-nonsense and start asking the following question to Linux developers: "Why doesn't Linux have DDK?" If some DDK appears, Linux will maybe (just maybe) become a more attractive platform for hardware manufacturers. I believe it'd be easier to convince "big players" to write Linux drivers conforming to DDK than to convince them to publish HW specs. Until such time, users are "doomed" to reverse-engineered drivers, "black magic" (like ndiswrapper), buggy (like nVidia's), or simply no drivers at all.

And I fully understand reluctance of ATI and nVidia to open up specs. Opening up the HW spec can reveal much about internal implementation. And internals are what they are living of. Encourages competition. And in the end, it's the users who benefit of it. (Just imagine ATI copying every feature of nVidia with same performance and comparable price, and vice-versa. They would simply loose any incentive to further develop their chips. At least until a newcomer to the market appears.)

Tags: linux nvidia drivers

Another critique of "free" software zealots

2006-10-07T21:09:00.000+02:00

This article announces a completely "free" browser named IceWeasel, derived from the Firefox code. This article points out some problems that distributions like Ubuntu and Debian have while distributing Firefox, and why Firefox might not actually be "free".

In my opinion, they present a skewed view of the matters, unfairly picturing the Mozilla corporation as a "bad guy". Quote from the 2nd article: "Though Debian and Debian-derived distributions such as the popular Ubuntu Linux currently include Mozilla Firefox, they do not typically include the actual Mozilla Firefox logo."

The question to ask is: WHY do they remove Firefox of its logo? What do they put instead? Logos have extreme importance in todays world, and can be actually said to sell products (look for example at Nike). What kind of ethics drives these "free" software developers? It seems that not only they want to own the code, they also seem to want to own corporate identity. And be able to remove it from programs at their will, possibly replacing it with their own. That's using hard work of another company to promote themselves. If this is kind of ethics that RMS and FSF stand for, their effort is better renamed to "slave" software.

Tags: fsf gnu mozilla firefox

The terrorists have won

2006-10-01T08:59:00.000+02:00

Here you can see a small video showing violent reactions of very small quantities (as small as 2 grams) of alkaline metals with water. Imagine sneaking 2 grams of lithium onto a plane, buying a bottle of water on the plane and dropping lithium inside it. KABOOM! Or better yet, drop it into the toilet. Even if it doesn't crash the plane, it'll make an unforgettable experience for the passengers. Are metal detectors sensitive enough to detect 2 grams of any metal? Can the security officer examining your hand-baggage through x-ray notice any object weighing 2 grams?

And the new EU airline security regulations, that will take effect from 1.11., forbid carrying more than 1 deciliter of own liquids onto the plane. Who are they trying to protect and from whom? Just to make it clear, I have no intentions of blowing up planes or killing people. This post is a form of protest, and a way to point out worthlessness of most of these security measures. Esp. forbidding liquids. If terrorists want to mass-kill people, they can do it almost undisturbed at check-in waiting lines.

As a side note, people die. Nobody lives forever. More people die of cancer than have died in terrorist attakcs since 9/11. Yet much more money is spent on "war on terror" (it'd be better renamed to "war generating terror") and fear-propaganda about dangers of terrorism than on people's health. I wonder how many people would stop smoking, eating junk food and began living healthier in general if that much money were invested in anti-smoking and other health campaings.

IMO, the way to fight terrorism is not to take away freedom from people and giving it to the government (exactly what is happening now in eg. US) and corporations (eg. airline security bodies). The word "terror" comes from the Latin language, and its original meaning is fear, fright. Given this meaning, and considering how many people are afraid and frightened, I think it's fairly OK to say that terrorists have won. Not only are the people afraid, certain governments seem to be pushing their citizens into dictatorship. Slowly, but surely. (Just look at the new "torture law" in the US.) Exactly the thing they claim to be fighting against.

So how should we fight terrorism? First, stop being afraid. (That might not be in the interest of certain presidents, as the fear they themselves have generated by their propaganda is the only thing keeping them in power). Second, we as a society should adapt. As the human immune system adapts to bacteria and viruses, the society should adapt to terrorism. As with diseases, there will always be random casualties. But random casualties are already all around us (home-accidents, car-accidents, drug overdose, medical mistreatment..); why do we have to single-out terror-accidents (I purposefully use the word accident here!) and make a fuss about them?

I don't have a recipe for the "adapt" part. People do not want to be killed by terrorists. People do not want to live in fear. People do not want war. I believe that people will cooperate on their own with police to prevent bad things from happening, only if given a chance. But as long as they are afraid, they won't dare take that chance even if given.

Good examples of the latter reasoning are arrests for attempted attacks in London and Denmark. That's commendable. But stricter security regulations are not justified. It's like fighting diseases by forbidding bacteria. It doesn't work.

Tags: airline security terrorism

How do commitees invent?

2006-09-28T14:50:00.000+02:00

This article is still a fascinating read, although it dates back to 1968. It analyzes some fundamental aspects of large systems development, and has some nice insights. Although I didn't read the book "The Mythical Man-Month", reading the article reminded me of the title.

Tags: business management organization

In defense of Tcl

2006-09-26T18:14:00.000+02:00

I found link to this article on reddit, and following the discussion, a link to this article. If you're short on time, I recommend you to read the 2nd one.

tcl programming

Category theory

2006-09-24T18:39:00.000+02:00

I have just finished reading the book "Conceptual Mathematics: A First Introduction to Categories". It is one of the best mathematics books I've read. Not only because the exposed theroy is very interesting (indeed, I've learned deeper meaning of some things that I've taken for granted until now), but also for its exceptional presentation style: concepts are explained through many illustrated examples in an accessible way, without delving into deep abstractions.

If you want to read an accessible introduction to category theory, I can heartily recommended this book.

Tags: mathematics books

How the questions shape the answers

2006-09-13T18:51:00.000+02:00

Many things today are advertised based on evidence obtained by polls.
This article [PDF], although a bit old (dating to 1999), is a fascinating read. Basically, polls can end up with dramatically different results, depending on the way the questions are asked. I'm wondering whether marketeers use these tricks as described in the article when constructing polls that should come out in their favor..

Tags: psychology marketing

A critical view on upstart

2006-09-09T14:10:00.000+02:00

There has been lately much fuss about upstart which is supposed to be a single replacement for several daemons: SysV-init, cron, inetd, hotplug... This article is a commercial trying to sell upstart, but somehow it hasn't convinced me.

The first reason I'm not comfortable with the idea is that UNIX is built on the philosophy "one tool for one job". Every tool should do one job and do it well. Merging several different tasks into one program just feels "yucky". It feels "windows-way".

The second reason is security and stability. Take for example cron. Even though it has a seemingly simple task, a very popular implementation, vixie-cron, had some security bugs in the past. Now it's going to be reimplemented again. Not to mention that upstart then becomes a single point of failure. Imagine e.g. remotely induced reboot or kernel panic by triggering some bug in upstart's networking code and making it crash. (And since it's running instead of init, it'll bring the whole system down).

Rest of this post is a dissection of the article cited above.

The first part of the article is what I call "Problem setting." In trying to explain why SysV init doesn't work today, the autor says "The simple answer is that our computer has become far more flexible." and enumerates certain situations which do not really pose a problem. Most of them are related to hotplugged hardware which is already handled (I see it working nicely on RH and SuSE). He concludes with "We've been able to hack the existing system to make much of this possible, however the result is chock-full of race conditions and bugs." While I admit that there may be some problems, saying "chock-full" would be a blatant exaggeration.

Question 1: Why replace replace everything instead of sticking with the UNIX philosophy and making the current system better?

The second part is "Design". On the surface it seems sane and well-designed, but take a look at the example list of events; the most striking one for me is "the root filesystem is now writable". He doesn't say who is supposed to generate these events! This is a shift of responsibility from getting the startup script ordering right to generating the right
events at the right time. Currently we have a small, well-controlled set of dedicated processes, and the upstart system seems to lead towards an explosion of possibilites along at least two dimensions: kinds of events and when they are generated.

Question 2: Who is generating events? Who is writing event handlers? If the event handling system is extendible, how is the system integrity guaranteed (so that the faulty handler doesn't bring the whole upstart process down)? What happens when an event isn't handled because a handler is missing? Is it an error, how is it reported and to whom, is it
simply ignored..?

The third part is "showing off" or FUD-ing. Showing existing tools in black light in order sell "upstart" better. This is the funniest part! Namely, the author doesn't seem to find good arguments against initng, a dependency-based system, so he resorts to ridiculous argumentation: "However this means that you need to have goals in mind when you boot the system, you need to have decided that you want gdm to be started in order for it, and its dependencies, to be started.", continuing with "[..initng] It can reorder a fixed set of jobs, but cannot dynamically determine the set of jobs needed for that particular boot." and finishing with "initng starts with a list of goals and works out how to get there, upstart starts with nothing and finds out where it gets to."

Question 3: How is the computer supposed to figure out, even before it is turned on, what the user has in mind and what should be the target configuration? How could it know that the user on a particular boot wants e.g. xdm to boot, without any user input (e.g. without being given a goal)?

upstart seems like a solution to an invented (or, to say the least, exaggerated) problem. I hope the author does better job of coding than argumenting its usefulness.

[From personal experience, dependency-based system is used on FreeBSD, NetBSD, and on Gentoo Linux. It's very easy to maintain, and I like it better than SysV-init style boot process.]

[Another note: One should distinguish between the init program and the SysV-init style boot scripts. It is possible to use the (SysV-)init program, with a dependency-based system. And that's exactly what Gentoo is does.]

Tags: linux upstart ubuntu

Deliberate bad engineering

2006-09-06T20:10:00.000+02:00

Today I discussed a simple problem with a colleague. He wants to design a simple format for representing graphs (nodes and arcs). He said that it's probably going to be XML-based to which I replied that it is a very bad engineering decision (see below for short explanation why and other choices). He agreed to that, but he's going with XML anyway. He said that today's IT industry is full of people "falling" for 3-letter acronyms and that he just wants them off his back. So, from the engineering viewpoint, the better solution has lost because of XML's "psychological effect". I imagine something like "It uses XML, therefore it must be good." Bullshit.

Problem with XML is that it's not very human-friendly and it's complicated to parse. Yes, you have ready made parsers, but you still have to walk the parsed tree. I suggested embedding an interpreted language such as Lua or Tcl. Syntax is definetly more readable than XML, parser is there, the user gets additional power (e.g. programatically constructing the graph instead of tediously enumerating nodes and arcs), and there is no "walking the tree". "Tags" in the scripting language can be bound to C functions and made directly executable, thus constructing the internal graph representation as the graph description is read, w/o subsequent walking of the tree.

The better solution is obvious, he agrees that it's better, but he's still not going to use it because of the "buzzword effect". How many projects have ended up taking the buzzword route, sometimes to their own detriment (I've myself participated in one such project.. I suggested otherewise, but the management decided to go the "3-letter way", and the project was more than half a year late).

Tags: software engineering programming xml

An idea for gmail

2006-09-02T19:00:00.000+02:00

How about revoking emails? Namely, the user sending something to @gmail.com could also send a special "mail revoke" message to revoke his sent mail. If the sender is also a gmail user, he could be offered a simple GUI option. To eliminate potential privacy concerns, the feedback would be either "revoke received" or "trying to revoke invalid message". Any other kind of message would let the sender know whether his mail was already read or not.

What would be the safe contents of such message? Something like SHA1(sender_address || SHA1(mail_body)) would be sufficient, although crude in the first iteration.

More importantly, mechanisms to securely "revoke" only own messages are available, why isn't such option already implemented?

Tags: google gmail

Rant and new stuff

2006-08-31T19:43:00.000+02:00

It's a long time since the last post. In the meantime i have been on vacation for 10 days, and at least 1.5 weeks w/o a decent net connection.

I have continued to work on PKCS#11 support for GnuPG. I just have to say that I don't like how the code looks like. On the one hand, a set of mutually cooperating processes seems like a good idea. On the other, it's almost impossible to debug when writing own extensions, and the lack of documented error-handling protocol doesn't help either.

On the up-side, I have written a small C tutorial oriented towards OS development. It is aimed for students enrolled in the OS course taught at University of Oslo and University of Tromsø. You can find it here.

Tags: gpg gnupg programming c

Prime sieves

2006-08-07T19:00:00.000+02:00

Here I have described a relatively simple way to cut down the space requirements for computing the Sieve of Eratosthenes.

Tags: mathematics programming algorithms primes

The plague of mailing lists

2006-08-05T16:59:00.000+02:00

Open source is plagued by mailing lists. If you want to report a bug, how do you do it - well, in most cases it boils down to sending a mail to the project's mailing list. It gets worse: you have to subscribe before you're allowed to post anything. Subscribing/unsubscribing to a mailing list just to report a bug is a major hassle. Recently I've reported bugs for two different projects[1] directly to authors, and haven't received any reply. Maybe they received it, maybe got eaten by their spam filtering, maybe they just don't have time. Bottom line: I don't care. If they want to receive bug reports from their users, they'd better find some more convenient way. For example, take a look at how gentoo handles it. Reporting a bug to the gentoo project was very easy, painles and I didn't experience it as a hassle at all.

[1] One of the projects is Ingo Molnar's linux realtime preemption patch.

Tags: open source linux mail gentoo

Switching to emacs

2006-08-02T22:07:00.000+02:00

I've pulled emacs 22 from the CVS, compiled it and switched to it from Xemacs. Main reason is that I want to use the org-mode (included with emacs 22) to keep collection of notes. Although it is supposed to work also with xemacs, i haven't been able to byte-compile it; it fails with an obscure error message. I did report a bug to the author, and I'm waiting for an answer now. I have a feeling that xemacs is now lagging behind emacs development, and it has definetely less community support on freenode irc. Moreover, it seems that most packages are first developed for emacs, and only then (maybe) ported to xemacs.

Tags: emacs xemacs

C++ std::vector problems

2006-07-23T15:30:00.000+02:00

Inspired by a usenet discussion, I have described some problems in the current C++ definition and implementation of allocators and vectors. You can read more about it here.

Tags: C++ STL programming linux

Few links

2006-07-20T21:40:00.000+02:00

Few days ago I stumbled upon this very nice and high-quality blog about math. This is another useful link with the FXT book and algorithm library. I'm amazed what some people generously give out for free.

Tags: mathematics programming books

Another GPL rant

2006-07-17T19:52:00.000+02:00

You might already know that I'm less than a fan of GPL. Here is another view on the issue.

Tags: GNU GPL FSF

New stuff...

2006-07-02T11:12:00.000+02:00

Soon I'm traveling to Portugal to attend the ICDCS 2006 conference. Related to this, I have put some new stuff on core-dump, my web site.

Now I'm a bit in between.. Maintaining a blog at blogger, and putting "more serious" stuff at core-dump. I've been thinking to continue writing the blog at core-dump, but somehow it doesn't feel "right". I like to keep the distinction between "serious" and "not serious" stuff. Any comments on that?

Tags: rumblings

Dangerous Javascript

2006-06-29T19:37:00.000+02:00

This article, titled "Knowing the User's Every Move...", is worrying. From the abstract: "In this paper, we investigate how detailed tracking of user interaction can be monitored using standard web technologies." In short, they have developed some JavaScript code (which runs in Netscape, Konqueror/Safari, IE and Opera) as well as proxy which transparently injects that code into page HTML before it is delivered to the client. This code enables detailed tracking of users actions including mouse movements, clicks and key presses.

This is particularly worrysome, as this mechanism can very easily be abused. Moreover, the current controls in, for example, Opera 9 are very inadequate. If I disable Javascript, then I can't use advanced AJAX applications, such as Gmail. On the other hand, there is no possibility to have Javascript enabled only for "trusted sites" stored in some list, and administered by the user.

Tags: privacy javascript security browsers