Tag Archives: survey

Smap Version 14.03 Forwarding

3 Replies

Smap version 14.03 has finally been released.  It should have been available from the end of March. The next release should be a small one and hopefully we can get back to the end of month regular release date.

The major new feature is forwarding of surveys.  Other changes are improvements to monitoring, support for larger result sets (paging) and a more laptop friendly deployment.

Monitoring

Totals

When you select “Monitoring” by default you will be shown total counts for success, errors, duplicates and “not uploaded”.

monitor

Totals are only available in Table view.  In order to view locations of uploaded data you need to first select the source as “Instances” then select “show as map”.

Filtering

You can filter the uploads (or forwards) that you view in the monitor page by their status. Success, errors, duplicates and “not uploaded”.  Hence you can uncheck all statuses except for “Error” in order to see which uploads in your project or survey were reported as being in error.  When showing instances rather than totals the last 200 records are still shown.  However if you apply a filter to only show errors then it will be the last 200 errors that are shown.

Paging

Paging is now available in the Monitoring screen and for the survey table view in the dashboard.  200 records are shown by default but you can then select a forward / reverse button to move forward and back through the results. These buttons are located at the top of the table and are labelled “>>>” for forward and “<<<” for back.  They will be disabled unless there are additional pages available to view.

Total Result Records

In the dashboard, when you view all survey results as a table, then the total count of surveys will be shown as well as the count of “bad” records. This wil be a faster way of getting a count of all surveys loaded than having to export to a spreadsheet.

Deployment

Changing URL of the server

Previously it was assumed that the address of the server would not change.  This allowed the server to get away with storing the full url of all survey attachments in the database.  However this assumption is completely untenable if Smap Server is deployed on laptops.  The server URL can now be changed and all the attachments will still be accessible.

Simplified deployment

Almost all the editing of files that had to be done during the installation process is now not needed.  The one exception is that the IP address of the server still has to be set in the install script.   This is only required for webForms and will hopefully be removed soon.  So, assuming you are installing on a clean Ubuntu machine, the new installation process is:

  1. Download and extract the release
  2. change directory to smap/install
  3. Edit the install script to add the IP address (temporary)
  4. Type ./install.sh

This will also now run the deploy script so once it finishes you shoulld be able to log on and use the server.

This release will be added to the downloads page over the coming weekend.  There will be another blog post to announce its availability.

Bug Fixes

  1. Error in retrieving languages on the audit page fixed.  This bug prevented auditing.
  2. If the first record in survey had an invalid geometry then export to shape files would fail. An example of an invalid geometry is recording only one point when a polygon or line was requested.

 

Forward Results to another Server

Leave a reply

Beginning in Smap version 14.03 you can add forwarding to a survey template.  This will result in each submitted result being forwarded on to the target server.

Key characteristics

  • Each forward link sends results from a single survey template to a single remote server
  • You can add multiple forward links to a template allowing you to forward results to many remote servers
  • Submissions are sent in one direction only. Hence if the target server is receiving results from multiple sources it will have a different view of the results than each of the sources
  • Forwarding links can be added to the target server as well. These can point back to the original source server but it ahs to be to a different survey on that server
  • Existing results will be forwarded if you add forwarding to a template that already has submitted results
  • Results will queue until there is a network connection before being forwarded
  • The target server can be another Smap server or any server that supports Java Rosa such as ODK Aggregate

Offline laptops

One way of using this forwarding feature is to install Smap on a laptop.  This laptop can be used to aggregate and analyse data collected on mobile phones in areas that do not have a reliable Internet connection.  You can upload the results from the phones to the local laptop over a wi-fi hotspot.  Once you connect the laptop to the Internet it can forward the results onto a cloud server so that they can be worked on by a distributed team.

Community Laptops

image (6)

Community laptops are an extension of the “offline laptop” concept.  Each community can maintain its local view of data collected on the phones.  However they can set up a forward link so that a consolidated view of all the data from all the communities can be aggregated.

Demonstration

The following video shows the forwarding of results from a Smap Server to an ODK Aggreate server.

How to add a forward link

  1. Create the same survey template on the local and remote server.  These templates can have different names on each server but should be the same template.  A good way to do this is to download the template as an XML file from one of the servers and then upload it to the other.
  2. On the source server, find your template on the template management screen and click the upward pointing arrow next to its name
  3. Click the button “Add Forward”
  4. Enter the url of the destination server
  5. Enter a user ID and password.  You should use a user id that only has privileges to complete surveys (enum) as the password is stored in the database. So don’t use your Admin user id!
  6. Press the button to refresh the list of available surveys on the remote server
  7. Select the survey that you want to update
  8. Save

You can monitor forwarded results on the monitor page by selecting the “forward” radio button.

Testing a long survey

Leave a reply

When creating long surveys for fieldTask you may want to try out a newly created, or modified section, that is a long way from the start. Its time consuming having to swipe past a lot of early questions and even worse if these have been set as “required”.  To quickly get to the right section you can use the “Go To Prompt” feature. Figure 1.

Start the survey in fieldTask or, odkCollect, select “menu” then “Go To Prompt”. A list of questions will be displayed which you can scroll through to find the question you want.

If you have repeating groups in your survey then this list of questions will initially stop at the repeat group.  To go further press on the downward arrow at the left of the repeat group name. Figure 2.

 

end2

Figure 3

Then press on the name that will appear at the bottom of the screen. The questions in the repeating group will be shown and you can continue to scroll down to find the section that you want to test.

Exporting Surveys to Openstreetmap

Leave a reply

I have added a new option to the Smap Server that will export a survey in OSM format. The following video shows this being used to add house numbers to Openstreetmap.

The survey from that video is defined in an XLSForm template reproduced below:
osmsurvey

This survey will collect the street name, odd or even, country, city, postcode in the top level form. It then collects the house numbers and their locations in a repeating group.  Some of the top level fields are copied into each house record using the “calculate” type.

I have adopted a couple of naming conventions for surveys that are destined to be exported to osm.

  1. Any question name that starts with an underscore, such as _country, is not exported.
  2. If the question name starts with “addr_”, then that is changed to “addr:”. The reason for this is that XLSForm did not seem to support the use of colons inside names but they are used in OSM key values.

I completed the survey twice to get the house numbers for a street called Jewell Crescent. When the results are exported the OSM file looks like this:

[code language=”xml”]

























































[/code]

After loading into Openstreetmap it looks like this.

This new functionality is in the source code now. It will be included in the next production release of Smap on Sunday 28th of July, after which the server can be downloaded from here.
Other Thoughts

  1. Currently option names map to OSM values.  This is quite restrictive as ODK does not allow these names to have spaces.  I will probably add an option to the export function that, if set, will use the option label to set the OSM value.
  2. You can export any survey to Openstreetmap.  Most of the time these won’t have been designed to be loaded directly as the demo was.  However it can still be useful to get the data into JOSM and then manually copy information into new OSM nodes.  For example if you have a survey with information on health clinics.

Connected Arabic fonts in Android 2.3

3 Replies

I’m working with a customer that is conducting a survey in Iraq using Samsung Galaxy Y Duos (android 2.3) and fieldTask. Later model phones (android 3+) worked well with Arabic but not these earlier versions. Even after rooting the phones and installing new fonts it still didn’t connect the characters.  See below a screen shot using Dr Ben Ox Arabic fonts which seemed to give the best results.
arabic2 (1)

Fortunately there is an Arabic reshaping library available from https://github.com/agawish/Better-Arabic-Reshaper/ which can be used with these old phones. I added ArabicReshaper.java and ArabicUtilities.java from this library to the odkCollect library that performs survey completion in fieldTask. I then modified the “QuestionWidget.java”, “SelectOneWidget.java” and “SelectMultipleWidget.java” files in odkCollect to call the “reshape” function when setting the text. See line 12 below.

[code language=”java” highlight=”11,12″]
protected void addQuestionText(FormEntryPrompt p) {
String imageURI = p.getImageText();
String audioURI = p.getAudioText();
String videoURI = p.getSpecialFormQuestionText(“video”);

// shown when image is clicked
String bigImageURI = p.getSpecialFormQuestionText(“big-image”);

// Add the text view. Textview always exists, regardless of whether there’s text.
mQuestionText = new TextView(getContext());
// mQuestionText.setText(p.getLongText()); smap
mQuestionText.setText(ArabicUtilities.reshape(p.getLongText())); // smap
mQuestionText.setTextSize(TypedValue.COMPLEX_UNIT_DIP, mQuestionFontsize);
mQuestionText.setTypeface(null, Typeface.BOLD);
mQuestionText.setPadding(0, 0, 0, 7);
mQuestionText.setId(QuestionWidget.newUniqueId()); // assign random id

// Wrap to the size of the parent view
mQuestionText.setHorizontallyScrolling(false);

if (p.getLongText() == null) {
mQuestionText.setVisibility(GONE);
}

// Create the layout for audio, image, text
mediaLayout = new MediaLayout(getContext());
mediaLayout.setAVT(p.getIndex(), “”, mQuestionText, audioURI, imageURI, videoURI, bigImageURI);

addView(mediaLayout, mLayout);
}
[/code]

This improved the display significantly:
SC20130708-172712

This Arabic version of fieldTask is available for download from http://dev/smap.com.au. There are other widgets that could have this functionality added and presumably the arabic text typed by an enumerator in a text field should also be reshaped. These additional changes will be made on an as needs basis. The preferred option is to get a phone that supports Arabic fonts natively which would be a better option than having to root the phones to install new fonts.