My first Android application (actually my second, the first was a commercial one) is released and published. The application is already downloaded more than 1000 times and that's OK for a national application. The name of the application is inherited from the excellent web site Temperatur.nu and is TemperaturNu. Well it's hard to find a god name when you want to implement a excellent idea.

Architecture.
The TemperaturNu application is built with several of the Android OS cornerstones that make up a rigid application. There are the MapView activity, together with a location view, because that's what give this application a twist. Then there are the service, handling data downloading from the web API. Nobody want to handle communication exceptions in the UI, right? I have bad experience in that area from my previous Android application and will not go down that road again!
To join those parts together we need a content provider for data abstraction and a broadcast handler for notification. The data provider also make it easy to handle the asynchronous data download. And...  that's it!

Details.
The TemperaturNu application uses the Android location API to determine your current position. The position is sent in a request to the Temperatur.nu API, and position and temperature for the nearest weather stations is returned together with some other meta data. This result is coded as xml and are parsed by the data download service. When the data is parsed it's sent to the content provider for storage and a broadcast intent is sent. All of this is done in a 'fire and forget' thread in the service that is started in the start server handler. It's essential to run this on a separate thread because if you dot do that the request and parsing will run on the main activity UI thread and that's not what we want.

In the main activity the received broadcast trigger a reload of data from the content provider and update  the items on the map overlay. Several overlays are used in the main activity to handle the display of location and items

The content provider just keep the data in memory because I judged the temperature data to be to  transient to store in a database. The most important role of the content provider is to decouple and abstract the data from the data down-loader.

Temperatur.nu web API
Erik at temperatur.nu have been kind to set up a public API (description here) which the application use for data collection. The temperature data can be requested either by geo position or by a station id. Up to five readings can be requested in each request and the response is coded in xml and structured lika a rss feed. The API can also provide a graph image of the last 24h readings from a station. This feature will be exposed by the next release of temperaturNu.

Passwords everywhere

I’m getting to old to remember all those passwords sprinkled all over the web!  Most embarrassing is that I can’t remember the password to my own private applications, but there is a couple though: local mail server, homeserver user and admin, local wiki and blog, several mail accounts to name a few.

So to remedy this plethora of passwords I have joined the OpenId movement and try to add OpenId login wherever I can. My DasBlog already have it but not my ScrewTurn Wiki wiki. I really like the architecture of ScrewTurn Wiki with dynamically loaded providers and it was fairly simple to write a new user storage provider for OpenId users and adding the DotNetOpenId control to the login page. Unfortunately I have not found a way to avoid changing the application when adding the OpenId login. The alternatives are either to add a new sign-on page with the open id login control or modify the existing login page.  If the login page is modified then I have to apply the patch on every update but I get a direct login link from the main page. Any way the application has to be re-compiled and I lose some of the benefit of loading a provider dynamically. But I have not investigated other solutions so there might be a simple fix for this.

Login page modifications:

Download the DotNetOpenId control from http://code.google.com/p/dotnetopenid/ and set up a reference to it from the web application.
Add the assembly import in the login.aspx page:
<%@ Register Assembly="DotNetOpenId" Namespace="DotNetOpenId.RelyingParty" TagPrefix="cc1" %>

Then the actual login control can be added anywhere on the page.
<hr />
<cc1:OpenIdLogin ID="OpenIdLogin1" runat="server" CssClass="openidLogin"
      RequestEmail="Require"
      RequestNickname="Require"
      RequestFullName="Request"
      RegisterVisible="false"
      RememberMeVisible="True"
      PolicyUrl="~/PrivacyPolicy.aspx" TabIndex="1"
      onfailed="OpenIdLogin1_Failed"
      onloggedin="OpenIdLogin1_LoggedIn"
      onsetuprequired="OpenIdLogin1_SetupRequired"
/>

When using auto registration at least email and nick name shall be required and the full name can be requested. Either the nick name or the full name can be used as display name. The ScrewTurn Wiki username will be derived from the identifier as described below.

Provider

The hardest thing was to adapt the way authorization works when using OpenId. The user shall NOT be authorized when logging in to the application. The user is already authorized; the application shall just set the access rights and eventually register the user.  User data like user name and email is requested in the call to the authorization service and are used to register the user. I used a trick to get the claimed identifier (http://myuser.name.myopenid.com/) down to the OpenId provider. First attempt was to use the identifier as user name but then the user name regular expression complained. Technically the user should not be edited but the name is verified when the user is brought up for group assignment. Anyway I needed a unique identifier for the user and want to use the claimed identifier.

Well the way I resolved it was to use the password parameter instead. When an OpenId user is created the identifier is modified to construct a new username: the http:// and .com part is removed and the dots replaced by spaces. This will give a user name like ‘myuser name myopenid’ and that is unique enough. I chose to automatically register the user if the login fails. The OpenId provider defines two default groups: OpenId_Users and OpenId_Administrators and can be configured to automatically add a new user to one of those groups. Users will be added to the anonymous group if they are not added to any of the default groups. 

Provider configuration:

My primary goal was to enable me to logon as an administrator to my wiki so I added a feature for automatically adding a user to a user group. The provider maintains two predefined groups’ wher users can be added with the following configuration:
addtoadmin=http://myuser.name.myopenid.com
addtouser=http://myuser.name.myopenid.com

Installation:

When the login page have been modified, or a new sign-on page added to the web application it’s time to deploy. Start up the wiki and log on as an administrator to configure the wiki for OpenId access.  Upload the OpenIdProvider.dll to add it to the framework and configure the provider if you want the automatic logon feature (see above).  Then go to the group administration page and set access rights for the two default OpenId groups. That’s it! Logout and go to the login / sign on page and login via OpenId.

Login problems

Sometimes during the tests I have problem to get the sign in to work and gets Login_Failed. The same thing happens on my DasBlog. Usually it works after a login/logout on myopenid. 

Disclaimer

This was not intended to be an industrial grade solution for OpenId implementation on ScrewTurn Wiki but it can be used as a starting point for them who want to add it to their installation.  The code is delivered ‘as is’ without any guaranties regarding security etc.
I got rid of another password to forget and can move on with next project
The OpenId source code and dll can (soon) be downloaded from http://minidev/download/Openidprovider.zip

Well this is a thing that I have been thinking about for a while: I want to use UTC times in the rrdb to be able to handle when time change from summer time (day light saving) to winter time. It must be better to always use UTC time internally and convert on data insert and fetch. That will take care of the exception I get when the clock is set back one hour and I try to insert 'old' data. I will also remove the conversion to and from epoch time and just store seconds from the DateTime class.

The problem is that I need to restructure the code in several places. I have never liked the connection between the classes where one class store a reference to a 'parent' instance like the Archive have a parent RrdDb and the ArcState have a reference to a parent Archive. Then the ArcState access members in the parent.parent instance like Header header = parentArc.getParentDb().getHeader(); I don't think that the ArcState should have any knowledge about the Archive and absolutely not about the relation between Archive and RrdDb!  Why not? Because I plan to use local time outside the RrdDb and UTC time inside and I must be sure that all access to time variables are converted correctly.

Example:

          
            public
            long getLastUpdateTime()
{ lock (sync)
{ return header.getLastUpdateTime();
} } 
        

Here I want to do a conversion to local time but in Archive there are code like this:

          
            long lastUpdateTime = parentDb.getHeader().getLastUpdateTime(); 
        

long lastUpdateTime = parentDb.getLastUpdateTime(); 

Well if I put the local time conversion in RrdDb.getLastUpdateTime() it will generate some nasty bugs that are hard to track!

But according to the text above I don't want the Archive class to have knowledge about the RrdDb to Header relation so the refactoring is correct! Thats why I want to remove all relations to the 'parent' object and supply all needed data as parameters in the method call. Anyway it looks strange to me that a RrdDb instance call into an Archive instance that call back, via it's parent reference, into the RrdDb instance to get some parameter that could have been sent into the call!

I'm not sure how to handle the changes in the Google code repository. I still keep the code in my private SVN repository and I will do the changes on trunk there. I will try to update the Google repository when I'm done.

Rrd4n(et) are public at last. After a long and painful time of ambivalence and un-determination I decided to publish my rrd4n lib at Google code. I hope I made the right thing and selected the right license type. Well time will tell. The project name on Google is rrd4net and not rrd4n because I manage to register rrd4n on my professional Google account.

And now the disclaimers: The software isn’t finished!

Especially the configuration tool need a lot of work until it is usable, but I needed a tool to set up the rrd databases and view the graphs during development. There are several menus that don’t work anymore and it’s not obvious or intuitive to use. There is one nice feature though: you can drag an rrd database from the left tree wiew and drop it in the graph definition area to make a quick test of the data in the register.

The separation of the data storage from the graphing is a bit over complicated. It’s nice to be able configure the data source in the app.config file but I’m not sure if it’s worth the effort. But it was really nice when I wrote a MySql accessor  (not in the Google code repository right now) to feed data from my MySql database into the graph. That experiment also made me realize why RRD is such a good idea: the MySql database contain temperature readings from 4 sensors with a resolution of 1 minute for about 3 years i.e. a huge amount of useless data with high resolution. It takes for ever to get the data out of the database. So now I have to write a program that feed the data from the MySql database into a RRD database.

Well right now I’m planning for an Android project so I will not spend that much time on rrd4n for a while.

Happy Round Robin

I have some plans to make the source code public in the near feature. The original java package was published under GNU Lesser General Public License and as I understand it it's OK if I do the same as long as the files contain the license text. I have to do some more testing and there are some 'technical debt' collected during the development. I think I will put the source code in a Google code repository when I'm ready.

At last, after a lot of refactoring and polishing the configuration tool is finished, or good enough, for now. I added some nice docking features (thanks to WeifenLuo DockPanel Suite) to enable a bit of plug-in behavior (more on that later). The docking view made it possible to show both database properties and graphs in a natural way. And as a frosting on the cake I added a drag and drop feature: a database definition can be dragged into the graph definition text box. The database definition then converted to a graph definition with a DEF declaration and a line definition for each data source in the database, a convenient way to do a quick view of the data in the database.

Database view:

Graph view:

I have spent a some time on my RRD code package. Porting the graphics and formatting, specially the date time formatting, was pretty straight forward but I have spent a lot of time on the configuration tool. I didn't realized that there was a fairly good implementation in Java until I was half way through! But my tool is fully visual (and still a bit buggy) and fully functional for setting up the database, import cvs data and test the graphing.

I have also made some serious refactoring on the core rrdb code, separating the storage part from the actual data access. I assumed that if an asp.net application should be able to access the rrdb files on another computer there could be some security issues! So I created a rrdb file server application where the graph processor could fetch the data by .net remoting (yes I know it's slow and over-complicated but it works). There are also a local file provider so I don't have to run on the server during testing. And naturally the right access provider is injected into the graph processor depending on the configuration file! The service has also a data store interface for data collecting that i plan to use in the data logger.

Talking about the logger it's up and running now. For the moment I collect data from my geo-thermal-heat-exchanger, or heater for short, where I measure compressor run time, hot water production and return water temperature. I need to get some more DS1820 sensors to measure brine temperature as well

Tonight I added the graphing to my rrd4n configuration tool and generated the first graph from the logged data. The logger have been running for almost 48 hours but the photosensor get out of position on the heat exchanger so there are some hour missing in the data. Still it's up and running!

Now I want to put this data on my webserver but I'm not sure how to get access to the RRD file from the web application. And before that I have to get te hardware ready so I can detect when the heat exchanger produce hot water. Some tasks for the weekend. 

Works done! Almost all code in rrd4j is ported to C# and I have run the tutorial examples. The trickiest part was the polygon drawing for bar charts. It took some time to figure out how to do it in GDI+ but in the end it was easier than in Java. After adjusting NaN values and adding missing segment at the end of the data series I just feed the segments into a Drawing.GraphicsPath object and call FillPath at the end. Works like charm!

There are still a lot of test to run on the package and I guess I can't resist to do some refactoring to apply some more C# style and remove the Java syntax that remains in the code. The refactoring will be done on a 'need to be done' basis.

So now I'm moving on to my logger project so I'll get some data to feed into the RRD