Monthly Archives: June 2018

Server Outage

I’d like to apologies for users of the sg server who experienced a 5 hour outage today. This should not have happened and was the result of deficiencies in my IT processes.

The number of surveys submitted to Smap hosted servers has been increasing over the last 3 years and is now averaging nearly 4,500 per day.

On the free sg server it is around 2,000 per day.  However on the 13th and 14th of June we received 11,897 submissions 80% of which included a high resolution image of a cocoa tree. This caused a big drop in the available disk space.

The IT process went:

  1. I received a text message that we were down to 4GB at 5pm UTC.
  2. Immediately ran some clean up scripts that freed up 9GB,
  3. I then added 60GB extra disk to the server.  However this disk is not made available until you reboot the server.
  4. The server was still being heavily utilised so I decided to wait until later in the evening to reboot when, given that most users are in Africa or Asia, the load should have been less.
  5. Then I forgot!

So instead of a 30 second outage we got 5 hours.  I will endeavour to ensure that this does not happen again.

Adding Artificial Intelligence to your forms

Single-layer feedforward artificial neural – By Akritasa [CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0)]

Smap Forms are a great way of implementing what used to be known as “Expert Systems”.   While you are collecting data with a Form questions can be displayed or hidden based on their perceived “relevance”.   In the same way recommendations and feedback can be shown based on the answers that have been provided.  If A and B then show C.

One reason that this is great with Smap is that the expert advice can be embedded easily in a system whose primary function is the humdrum capturing of information.  It is not necessary for the user to sit down for a dedicated session of receiving expert advice.  Another reason is that a domain expert can create these forms without needing input from a programmer.

However these rules are a long way from Artificial Intelligence.  Once your form has collected the answers to 20 plus questions as multiple choice, text, audio, images and video then there may be critical information that you want to feedback to the user immediately hidden in that data that you cannot extract with a few “if” statements.

Version 5.5 of fieldTask (available from the google play store) adds the ability to call a server passing it data you have collected and to get back a response that can be used to guide the next steps.  You need to be filling out the form on line of course and the service you call need not be an Artificial Intelligence web service, it may just look up reference data, however we see the use of AI services in this way as particularly compelling.

You can try this out in fieldTask by adding a “calculate” that calls a lookup_image_labels() function.  So in an xlsForm your questions might look like this:

type           name                      label / calculation
image        scene                      Take a photo
calculate   scene_objects        lookup_image_labels(${scene})
note           show_objects        Objects in photo are: ${scene_objects}

And this is what the above form looks like on the phone.

Firstly Take the Photo

And here are the objects that were identified

Well this is not actually very useful!

The AI service called by lookup_image_labels() is the AWS Rekognition service that can identify things in photographs such as cars, people, computers, desks etc.   Even that task is done far from perfectly, it did not identify the  bowl, unwashed or otherwise.  And it will probably identify nothing that the person filling in the form could not identify for themselves.   This information would be useful on the server in order to search for images  but you can already generate these labels automatically, using Smap, once the form has been submitted.  It hardly seems necessary to make the person completing the form wait for these labels to be returned from the server.

So why do it?

Well actually the Rekognition service is just an example.  We can add calls to your custom AI engines that can identify patterns in the data or images and that may be critical in identifying the advice to be provided or the further information that needs to be collected.

Smap can be used to collect the data required to train an AI engine and now it can use the decisions from that AI engine in collecting more data creating positive feedback to add value to your processes.