Wednesday, November 30, 2005

Siemens Success!

My colleague, Andy Piper, hinted in the comments to a previous post that merely getting my contacts file to display in an emulated Siemens SX1 was unimpressive and that a real techie would have written some code to transfer them from the emulator to my new phone.

It was a ridiculous challenge - automating a task you will perform once is madness - but any suggestion that my consulting and travel has dumbed down my technical abilities must be countered. And countered AT ONCE!

Accordingly, I wasted invested my evening yesterday in solving this problem. You could argue that there are better ways to spend a Tuesday evening. And I would agree with you. However, the time is gone forever so let's press on....

The lazy reader can quit reading here if they like: the bottom line is that I have managed to take my unreadable contacts.cdb file and import it into my new Nokia 6021 without having to do any manual retyping of contact information. Hurrah! Yay for me. &c.

For the more inquiring reader, here is how I did it.

Firstly, I would like to pretend that I analysed the problem, planned an approach, sized the work involved and methodically reached a successful conclusion. However, that would be a lie. Rather, I made egregious use of google, grep, sample code, hacking, guesswork and luck. Nevertheless, the problem can be broken down into several areas:

1) Writing a program to run on the phone emulator to interrogate the contacts database programmatically

2) Using the program from step 1) to write the data into a form that could be extracted from the emulator

3) Some technique to take the output of step 2 and turn it into something that the Nokia sync software understood (e.g. a Windows Adress Book file)

4) Sync of the data onto my phone.

These problems each presented their own challenges. I don't pretend to be proud of the solution I came up with. But it worked. Which is important. Of course, if I'd done this for a client, I should have been sacked.

So, firstly, how did I interrogate the contacts database? My language of choice (Java) was quickly ruled out as I couldn't get any of the sample java applications to run on the emulator - I simply couldn't get them to install :-( The SX1 emulator seemed incapable of loading .jar or .jad files. When I was seriously considering hosting them on a web server and configuring the emulated phone to download them over a simulated WAP connection I knew I should change approach.

So, instead, I dusted off my trusty copy of Microsoft Visual Studio 6.0 and followed the instructions in the Symbian 6.1 SDK on how to get the "HelloWorldPlus" application to run inside Visual Studio. Very neat - and it actually worked!

I now had a sample C++ Symbian Application that I could modify.

I found another PDF (shipped with the SDK - or perhaps the SMTK) that gave some sample snippets on how to drive the CContactsDatabase. Apart from one mis-step (you have to regenerate the Visual Studio workspace if you add any dependencies on other libraries to the .mmp file), I quite quickly had an app that would iterate over my contacts and display them in a pop up window on the device. Excellent!

Not so fast..... this was, it turned out, the easy part. The hardest part was that I couldn't find any nice way at all to write to a file in any useful way. Conversions between 8-bit and 16-bit string representations, my lack of knowledge of the Symbian primitives and more meant that I got bogged down here. (Not having written any C++ for five years also didn't help).

Eventually, I discovered how to create a useful character array on the heap and, after more digging, how to copy individual strings into it. I used this knowledge to write my contacts into a contiguous chunk of memory (with various tags and delimiters to separate contacts, field names and values)

Hurrah! You see: I am a hacker at heart so once I had everything in a contiguous chunk of memory I was done. I simply placed a breakpoint at the end of my program, checked where my buffer was located on the heap, viewed that piece of memory and used copy/paste to get the data into Notepad. (Yes: it's horrible... but it worked....).

So now my problem was one of search/replace and some nifty perl to generate a useful CSV file.

A bit of tweaking in Excel and I had a CSV. I used Outlook express to turn it into a .wab file. I could then sync it to the phone.


The question, of course, is: why did this need to be so difficult? I return to my original statement on this whole saga: open standards are a beautiful thing. Siemens/Nokia/Symbian should be thoroughly ashamed of themselves.


andyp said...

Richard, I'm glad I caused you to exercise your technical skills. It's an all-to-infrequent event these days... ;-)

Completely agree that it is unreasonable for these things to be so hard. I spent some time playing with SyncML a couple of years ago, which is a standard, but trying to get my (then) Nokia 7650 talking to a SyncML server running on Linux was itself a challenge. It does seem that several of the phone companies are intent on making things difficult for users :-(

Richard Brown said...

"t's an all-to-infrequent event these days... ;-)"

Yeah, yeah, yeah. What-ev-ah!

As for making things difficult... I suspect that engineers and managers in phone companies suffer from an excess of optimism.

They seem to assume their phones will never go wrong and that, in any case, their "sync" software will work.

As anybody who has ever tried to use such software will testify, they are usually very buggy and suffer terribly from the fact that bluetooth drivers - even today - are still pretty shoddy.

Oh well - it's all sorted now.

The really embarrassing thing is that this problem started because Siemens could only sync to Microsoft Outlook... which as an IBMer I don't have a license for.

Except... it was only when I hit the need for Visual Studio 6.0 that I realised that my MSDN subscription could have provided a copy of Outlook all along :-(