Visual Studio 2008, take 2

Recently I've been extensively working with multithreading in C++, with pthread API, so the code is not portable to Windows. Since I want to get some experience with Windows programming and Visual studio (my holiday project :)), I decided to try out Boost.Thread, which is a portable multi-threading library.

So I downloaded Boost 1.37, the bjam executable for NT, read through the boost getting started manual, compiled the necessary libraries in 32-bit and 64-bit mode as well as a small test application. The process has been surprisingly painless -- it took me about one hour from downloading bjam to having a working test application that comes along with Boost.Thread.

Another reason for wanting to have a portable application is that it is painful to develop over a remote connection to unix server. A compromise solution, which I've been practicing lately, is setting up shared folders with Virtualbox, editing the code with XEmacs, and compiling/testing in the virtual machine. This kinda works, but does not make the code portable :)

New CSPIM release

I have released v1.0 of my CSPIM MIPS simulator. This release includes some previously outstanding issues, and the simulator can now simulate itself.

CMake and assembly sources

I've been trying to figure out how to "nicely" integrate building assembler sources with CMake. After some digging around, I found a nice recipe recently posted to CMake mailing list:

set_property(SOURCE cputorture.s PROPERTY LANGUAGE C)

This works because the C compiler, at least on UNIX, recognizes .s as assembler source extension and invokes the proper assembler command.

NetBSD 4 dissapointment

Well, I installed 32-bit NetBSD 4.0.1 in virtualbox. Then I attempted to install subversion through the pkgsrc system, and.. became thoroughly dissapointed. It pulled not only subversion, but also full apache (although it needs only the apache portable runtime, APR), perl 5.8, ruby 1.8, and who knows what else. I got tired of ruby compiling its "RI", whatever that is, and just deleted the VM. I think I'm going for some desktop ubuntu install now. Or maybe Fedora 10.

Acrobat Reader 9 annoying accessibility feature

The annoying feature being accessibility which turns itself on, and there's no option to turn it off. Sure, there are a bunch of accessibility preferences, but no "disable" option. So every time I opened a document, I got an annoying dialog that the "document is being prepared...".

Solution: find "Accessibility.api" and "ReadOutLoud.api" files in Acrobat's installation directory (under plugins) and rename them to something else. I have created directory named "disabled" and moved the files there. Works like a charm.

R

Recently, I had to process and plot a significant amount of data. Previously, I used to use gnuplot for that, but I've discovered two shortcomings: 1) gnuplot has limited data manipulation facilities, so I had to do data processing with perl scripts, 2) it has very arcane syntax and it's rather hard to make it do what you want, especially when you have slightly complex requirements (eg. bar charts of several nested data sets).

I have resisted learning R for a long time, because it also uses a rather arcane language with many complex and rather badly documented data structures. However, a week ago, I remembered my last experience with gnuplot, and I gave R a try. I'm glad I did even though it took me some time to get used to its data manipulation capabilities since the learning curve is rather steep. However, once you get used to it, it's so much easier and faster to manipulate data than with custom-made scripts, and much simpler to produce well-looking plots. Visit its homepage and take a look.

Windows Vista and Visual Studio

I've bought a new laptop, got 64-bit Vista business with it, so I said - heck, let's give it a try. After one month of using Vista, I don't miss linux desktop at all. I have power management that works all of the time (and that locks the screen after wakeup, without any special configuration!), Flash that does not crash at least once a day (crash on linux manifested most often in the flash plugin eating 100% of CPU), sound system and mixing which works out of the box (no need for mysterious sound daemons that will enable multiple sound applications running simultaneously), plus availability of software for gadgets (garmin gps, nokia phone). I have even found a nice and free virtual desktop manager (VirtuaWin). One month after using Vista.. I won't be installing linux on the laptop anytime soon :-)

You have probably heard that User Access Control (UAC) is PITA, and there are a bunch of cookbooks on the net instructing on how to disable it. Coming from UNIX background, I find it rather natural that you have to type in admin password before performing system-critical tasks. Suggesting to people that they turn off UAC is rather irresponsible, and I have not done it. I like being warned when some operation requires elevated privileges. Oh, and I have turned off the sidebar - gadgets there are mostly useless and they just increase the startup time (which is, I must say, rather decent).

I have found replacement for all of the tools that I used under linux, except one -- development tools. And now, my comparison standard is not linux, but Solaris, with Sun's native developmet tools, CMake and dbx. (BTW, Solaris dbx just rocks! gdb is a poor match for it.) I have installed Visual Studio 8 Professional (I have free license through MSDNAA), and it has several shortcomings in comparison with Solaris tools.

First, I've heard many windows developers say that VS debugger is one of the best, but I find it rather mediocre compared to dbx. VS debugger does not recognize C++ overloaded iterators (so you can't persuade it to show you *it where it is some C++ iterator class) and it is a pain to display deeply nested data structure (just print -r in dbx). VS has even its own embedded command-line interface, but nothing indicates that these features are available through it. Sure, VS debugger is nice for single-stepping through the code, but this is not the way I usually debug the code. (What usually happens in my debugging session is that some assertion/check triggers a crash, I inspect the core dump, put a breakpoint near the offending place, and quickly print few variable values.) On the upside, the VS debugger understands STL containers and can display them in a nice form. But I'd go for dbx any day, only if it were available for Windows.

Oh yes, changing command-line arguments through IDE for debugged programs is PITA :-) If I run the program from the shell and it crashes, it takes several seconds to drop into the debugger, which is kinda annoying.

Another complaint is that VS does not seem to come with a decent profiler such as gprof on Solaris. Most people recommend to use Intel VTune, which costs money. Sun has recently released (free!) SPOT tool which produces a nice report of hot-spots over several runs of your program, in the form of cross-linked HTML pages and graphs.

Then there's DTrace which is currently unique to Solaris and some BSD variants (OS X, FreeBSD?).

Lastly, project configuration management is easier with CMake. I have not yet figured out how I can persuade VS to inherit project properties from the master "solution" properties. Given that projects have parent projects, I should probably play with a deeper project hierarchy. (But why can't I set master settings in the "solution"?! Bleh.)

IntelliSense is nice, but.. it doesn't work all of the time (I see rather often a message in the status bar that IntelliSense couldn't find a completion.. ). Though, when it does work, it sometimes saves me a lookup of member name.

In conclusion - Vista is a rather nice environment for everyday work and it is very stable. However, Microsoft's C and C++ development tools are no match to Solaris's tools, at least when it comes to lower-level stuff. (Maybe not so surprising, UNIX is traditionally C and C++ oriented environment, while MS is moving towards .NET). I'm definitely somewhat less efficient in development with VS (though not drastically); I don't know (yet) whether this is because I haven't learned all the features, or because Solaris tools simply are superior and have no match in functionality in VS.

New site

You might want to literally waste some time on my new website: riddleschannel.org. The purpose of this page is to conduct certain experiments. If you want to help, just spread around the link!

New code release: ext2frag

A master student of mine has developed a FUSE filesystem as a part of his master thesis. He needed to compare several fragmentation metrics of his own FS with that of EXT2. However, no decent programs for calculating EXT2 fragmentation statistics are available, so I set out to code one since the student was really short on time. You can get the program here.

Free society ; some tips

I've recently watched Lessig's presentation on free culture.  It has been recommended to me as an example of interesting use of slides, but it is also an excellent lecture on how copyright stifles innovation and development.  The following is the basis for the whole lecture:

1. Creativity and innovation always builds on the past.
   
2. The past always tries to control the creativity that builds on it.
   
3. Free societies enable the future by limiting the past.
   
4. Ours is less and less a free society.

There's another noteworthy quote (by JC Watts, a US congressman/senator?): "If you're explaining, you're losing."

Regarding tips: perl's CPAN shell is an excellent tool to install packages and their dependencies. However, it uses active FTP by default which is more and more dysfunctional because of firewalls.  Exporting the environment variable FTP_PASSIVE=1 before running will make it use passive FTP.

Few rants...

First: I'm switching from emacs back to vim. It's nice to have everything integrated into emacs, it's calculator is especially good (reminds of HP48), but I've found emacs nearly impossible to use for C++ coding. Namely, its internal parser gets often confused, and it won't let you type a simple colon (':'). Usually closing the file and opening it again fixed the problem, but not few days ago. Emacs simply refused to allow me to type in ':' (as in 'public:'). That was too much for me, and I switched back to vim (I've anyway used vi emulation in emacs; i find concise vi-keystrokes much more accessible than emacs' endless strings of characters in combination with alt/meta..)

Second: ubuntu sucks! I have it installed on my desktop machine on work, because that's "what everyone else uses" and only ubuntu CD's are available for installation. Today I ran the latest upgrade of 8.04, rebooted the machine and I was greeted with half-functional GNOME. I.e. the gnome-panel was missing and had I not had some folders on desktop that allowed me to get access to the file manager and eventually run an xterm, it would be completely unusable (i.e. none of the keyboard short-cuts worked either). Solution? I have no idea -- I installed xfce4 to get a functional desktop. The good thing is that xfce4 does not display the annoying update notification every now and then :-D Anyway, ubuntu owes me compensation for 1.5 hours of lost work time. Talk about "no-cost" software :-P

To hell with climate change, time to speak up for the fish!

Most media are writing about the polar ice melting, polar bears being exterminated.. like anyone's going to notice that they're gone. At the same time, something much worse is happening, and it should deserve at least as much media coverage as climate change: unsustainable overfishing in the seas. This article is a very awakening and scary read. Spread the word and make politicians do something! I've never seen a polar bear, nor do I have a particular desier to see it, but i like very much to eat fish.

[PS. I think that disappearance of any species is regrettable, but I certainly prioritize a food source over some exotic animal. It's regrettable that media don't have same opinion.]

Sudoku solvers and cairo graphics

I have decided to play a bit with constraint programming, so I'm rewriting my logic game solvers as constraint programs. For that I'm using the minion constraint solver to which I interface through python; I wrote a simple parser of minion's textual output. I wanted some pretty-printing, and doing it in purely textual mode has shown to be rather tedious to code. So I turned to the excellent cairo graphics library for drawing solutions. The library is surprisingly simple to use, and also comes with python bindings. Now I'm working on another puzzle, and after cleaning up the code a bit, I'll upload the new package.

Dent game: an example of nonexistant security

In Norway, there's a commercial campaign (ending 15.4.) for Dent candies; you're invited to play a game (www.dent.no), and the best scores win some prizes (the coolest prize is a free cinema-ticket for two valid for 1 year). Since the prizes are attractive, I was wondering what kind of security they implemented. Surprisingly (or not; Norwegian companies have a bad track record with security issues), the answer is none!

Before submitting my result, together with my data (email address and phone number), I fired up wireshark and found out the following:

• They first connect to stats.indexstats.com to report some basic statistics (a cookie with base-64 url, among other stuff, is sent and an empty GIF is returned).
  

• Then a simple POST request is sent with the score and entered data about the user. Everything through ordinary HTTP, and everything in cleartext! Some binary data is returned, but I haven't bothered to figure out what it is.

So it's trivial to record the traffic into a file, change the score, and resubmit the better score by e.g. netcat. (No, I haven't done it myself -- the hardest part seems to be guessing high enough to win, but still plausible score.)

What could have been done differently? It's impossible to secure an application 100% when the user has full control over the computer, but I think that just AES-encrypting the sent data with a key preshared between the client and the server would have risen the hacking bar significantly. You'd have to download the raw flash file, try to find the encryption key inside it, guess the encryption algorithm, etc. If the encryption key is not stored contiguosly in the file, then you'd also have to reverse-engineer the flash code to see what's going on.

In any case, very badly done web application.

Inkscape

I've been looking for a long time for some visio-like vector drawing program for Linux. Well, I've recently tried Inkscape, and, while still not a Visio replacement, it does its job pretty well even for technical diagrams.

MIPS CPU simulator, CMake

Today I've published the first version of a MIPS CPU simulator; you can read more about it here. Even though the version number is 0.9, the program is not in any case beta: it is quite stable, CPU torture tests pass in 32-bit and 64-bit environments, and there seems to be no open security holes, though error "reporting" is sometimes unfriendly: the simulator shall call abort() if the simulated program tries to access memory outside of its allocated range.. but I think that's still way better than doing random stuff to random memory. Work on better error handling is the top priority right now.

I'm also using this opportunity to learn about the CMake build system.

IT Underground 2008

I have again attended the IT Underground conference in Prague, where I gave a short lecture about PKI in practice. Soon I shall also upload the slides from the talk.

MIPS CPU

Things have been pretty hectic here, so I haven't had much time to post anything. Recently, I've begun a new personal project: writing a simulator for the MIPS I CPU arhcitecture. I chose this particular CPU because it's the simplest architecture for which gcc can generate code. There are many other MIPS CPU simulators out there, but all of them do more than I need, so I decided to write my own as simple as possible. The ELF loader and symbol lookup are finished, and now I'm working on developing test vectors for the simulator (a very boring part).

As for why I'm doing this - the answer will hopefully be clear soon :-)

Java leaks memory

Now, this is something to laugh at: when first launching Java, Sun boasted about garbage collector and that programmers will never again have to cope with memory-management problems. Fast-forward to 2008. to read Sun's presentation on avoiding memory leaks.

Some regexps

I have started to develop some CRM114 scripts to classify Usenet articles. Internally, it uses the TRE regexp engine and I needed a regexp splits article into headers and body; the separation happens at the first empty line. So here's the regexp; it seems to work :)

/((?:(?n:.+):*:_nl:)+):*:_nl:(.+)/

Some peculiarities: the :*:_nl: is CRM114 literal for newline, and the ?n: is a TRE flag to tell it that dot (.) should not match the newline character (by default it does, and this makes TRE different from PCRE regexp). The headers will end up in the 1st matching group, and the article body in the 2nd.