Tag Archives: fieldTask

Release 18.6

Leave a reply

Email Tasks

The ability to send pre filled or blank forms as links in  emails out to people for completion in their browser.  Capabilities include:

  1. Forms can only be filled in once by each recipient (unless the repeat flag is set)
  2. The URL to access the form includes a key so that the user does not need to have an account on the server
  3. You can upload email address and form names from a file and then have the emails sent to all of those addresses
  4. There is an unsubscribe link in all emails
  5. People can resubscribe, if they change their mind by going to the subscriptions page (on the home page select subscriptions)
  6. Emails with a pre-filled in form can be sent automatically when a submission is received for any other form

Really what this does is enable you to extend you data collection beyond the professional data collectors who have a logon on the server.  Hence you can do email surveys, as per Survey Monkey and also integrate input from external people in a multi step process.  So in fact Smap goes way beyond the capabilities in Survey Monkey.

Note: Email tasks are disabled on the sg server by default. Contact us if you want to use this feature.

Auto sending email tasks

You can also assign tasks that include previously collected data to an email address .   As with other tasks, that are completed by logged in users,  you can create these from previously submitted results or results as they come in.  Here is a video of an example process but there are an infinite number of ways you can use this feature:

After completing the steps shown in the above video and then doing an XLS report I got the following output:

Loading Email Addresses from an Excel File 

Email tasks are managed from the tasks module,  From there you can create a task group and then click on the file menu to download an XLS file.  In this case for new task group it will be empty but you can specify the “tasks” here.

The columns with blue headings are where you put details about the form.  (See above). The columns with yellow heading contain details about the person being sent the email including of course their email address.  You can add multiple email addresses to be sent a single form hence the greyed out cells in the above example where we have one form but two email addresses.  Also you don’t need to fill in the URL column, this is only filled in when exporting in case you want to manually send the links out without using Smap’s email manager.

Once you have filled in your forms and email addresses.  Click on the Menu File > Load XLS.  Emails will not immediately be sent, you will then have the opportunity to review the loading emails and when you are ready you can select the menu “Email Unsent”.   This will send the emails with blank forms.

Unsubscribe From emails

This became urgently needed due to the addition of the “Email Tasks” functionality, however the unsubscribe link is now added to any email sent from the Smap server including for example an email adding you as a registered user.  If someone clicks on that link they are asked if they want to confirm the un-subscription.  If they say yes then they won’t be sent any more emails.   They only way to resume sending emails to an unsubscribed address is if you can persuade them to:

  • Goto to the server home page
  • Click on the subscriptions menu
  • And subscribe again

Updates to FieldTask version 5.5

Well this was released some time ago but I didn’t make an announcement on this news site so here it is.  Lots of new features here contributed by the folk from ODK as well as the fieldTask developers. Some of the main ones are:

  • Online lookup of data from a server while the user is completing the form. You can lookup reference data, choices for a select or query an AI web service.
  • Non printable characters in barcodes will now show as spaces rather than being removed completely
  • Includes ODKCollect 1.15
    – Can set a dark or light theme to change the screen appearance
    – Add a “file” type question that allows you to attach any file to the form

 

 

 

version 17.08

Leave a reply

This release adds support for odk parameters.   Actually currently there is only one parameter supported and that is max-pixels which can be used to reduce the size of photographs taken in a form.

Usage

Open the form in the online editor and select “Parameters” as the property you want to change.  Then find an image you want to reduce.  Then set the max-pixels parameter to the size you want.  For example:

max-pixels=500

The number is the maximum length in pixels of the long edge of the image.

Notes

  1. As well as SmapServer 17.08 you will need fieldTask version 5.06
  2. There is no support in xlsForm for parameters yet hence you will need to enter the parameter in the online editor as described above.
  3. You can consider parameters and appearance as doing very similar things .  In the past appearance was used for many of the capabilities that odk will now be adding using parameters.
  4. Help is available in the reference guide.  https://docs.google.com/file/d/0B5_SmpWlQYxvWlBoWkdiT3FrY1U/edit

 

version-15-03

2 Replies

This release is primarily about adding support for iPhones and iPads when they are using Web Forms.  You can now take photos, and videos with your iPhones when completing smap surveys.  I’d like to thank Zarkman for donating an iPhone to Smap Consulting in order to help with the testing of this new capability.

zarkmaniphone

A user changing their settings on the Smap Server using the Zarkman donated iPhone

New Features

iPhone and iPad support

You can now complete surveys on your iPhone and iPad using WebForms. These surveys can included images, video and GPS coordinates.

If you conduct surveys that don’t take pictures then they can be submitted offline and will be stored until you have a network connection. However, when using an iPhone, if you take pictures as part of your survey then you will not be able to save the results as draft.  Also if you are offline when you submit, then the results will not be stored to be submitted when you are back on line.  You will need to stay in the web page until you get a network connection and then re-submit.  You can open multiple pages in order to complete multiple surveys, with images and offline, while using an iPhone.

To complete a survey on your iPhone, use Safari to go to your server. Then select WebForms and finally select the big blue button for the survey you want to complete.

This capability was in large part enabled through upgrading to the latest version of enketo-core as well as being assisted by the sponsorship of Zarkman.

Grid layout for Web Forms

Thanks to the the new version of enketo-core you can layout your web forms in a grid.

grid

An example grid layout from enketo running in web forms

To turn your survey into a web form grid you need to:

1. Add a settings sheet to your xlsform containing

style
theme-grid

2. Add w1, w2 etc to the appearance columns of questions that you don’t want to appear full width.

This example form from enketo has more detailed information.

Email Notifications

The email address to which to send a notification can now be obtained from the survey results.  When creating a notification specify the question from which to get these addresses.  This can be a select one, select multiple or text question. They question can be in a repeating group in which case all the emails entered will be used as the “to” address for the notification.

To specify multiple email addresses in a text question separate each email address with commas.

Getting email addresses from a select_multiple question

The names of choices in a select multiple question are used to create the data base table that stores the results.  These database column names cannot contain ‘@’ or ‘.’.  Hence for select multiple questions you have to modify the way the email address is specified.

  • Replace ‘@’  with ‘_amp_’
  • Replace ‘.’ with ‘_dot_’
emails

Choices sheet to specify notification emails addresses chosen by select_multiple

In select_one and text questions you can simply write the email as normal.

Email Message Text

You can now specify the text to be included in an email notification.  This can be specified for each notification.  Alternatively a default email text can be specified in the organisation settings.

Specifying the email domain

The email domain to be used for emails can be specified at the whole of server level in the “server” table.  Alternatively it can be specified for the organisation in the organisation settings.

PDF Generation

You can attach a PDF of the results to an email notification.  A PDF will be automatically generated using the same grid layout as is available for WebForms.  However in this release its still very much beta code.

More usefully a template for the PDF can be specified in the surveys settings.  Go to the survey editor and click on the “File” menu and then “Settings”.  Fields in the template that have the same name as a question will be populated with the results of the question.

PDF’s  can also be downloaded from a table view of the survey results. Goto the analysis page and create a table containing the survey results.  A menu button will be shown to the left of each record.  If you click on that you have the option to download a PDF or open a WebForm containing the record.  Note this feature is only enabled if the ability to edit surveys is specified in the organisation settings.

Miscelaneous

  • Files in the /smap/temp directory are now deleted each time you upgrade
  • If you rename a survey or delete it and then restore it then you can still download the XLS form that was used to create that survey.

Bug Fixes

The following issues have been fixed.

  • The default language could not be specified in the settings for the survey.
  • If you had two levels or more of repeating groups then when you exported results an unnecessary row containing data from the main form would have been added to the results.
  • Could not add media to questions using Firefox
  • Images of type “ico” were shown in the dashboard with an audio thumbnail
  • Filter values used in cascading selects were not allowed to have spaces.
  • When exporting surveys with more than one “/” in their name then the export failed.

Update (Version 15.03.02)

I have updated the released code to address some relatively minor bugs.

  • Notifications for soft deleted surveys were still being shown in the notifications list
  • The xls template for editing forms has been updated to include the settings tab and a column for the “style”.  You can use this to format web forms in a grid.
  • The link to this blog from the server home page was pointing to the release notes for version 15.02

Update (Version 15.03.03)

Updated for a very significant bug.  The web forms code was not included in the previous release!  So if your web forms was looking a bit flat and not responsive to relevant conditions then this would be why.  Installing this the .03 version should fix it.

 

Managing School Enrolments with Smap

4 Replies

Sometimes your forms need to change quickly.  One such case was suggested by World Vision staff who are working in Syrian refugee camps.  They wanted to record school attendance however new children are arriving daily and hence the attendance roll would be also changing daily.

Putting choices in a CSV file separate from the form can be the answer.   The file name is specified in the “appearance” column.  In the example below the choices will come from the “children.csv” file.  The other parameters to the “search” function are filters so that only children in “class1” are included.

enrol1

Select Question Getting its Options from a CSV File

Choices in external CSV files have been available for a while, however the January 2014 release of Smap will have the following enhancements:

  • Support for select_multiple.  Previously only select_one worked,
  • You can now analyse select questions with external choices in graphs, tables and maps on the analysis dashboard,
  • and, CSV files can be attached to a single form or shared by all forms in the organisation.

Demonstration

This demonstration shows information on new school enrolments being collected using a form and then being added to a CSV file.  This file is then used to record attendance in school and is also used by a separate form to record vaccinations given to the children.

The data can be used in multiple forms because it is loaded to the “Organisation” tab in the media page rather than the “Form” tab. If you loaded different files with the same name to “Form” and “Organisation” then the file at the “Form” level will take precedence.

The files for this example can be found here.

Process

  1. A school teacher uses fieldTask and a form called “enrolments” to record school attendance.  Any new children arriving that day are recorded in a sub form.  (using “begin repeat”)
  2. The back office team copy the details on the new children and put them into the “children.csv” file.  They upload this file to the Smap server where it replaces the existing file.
  3. The next day, before recording attendance, the Teacher presses “refresh” in fieldTask to get the latest form. They can then mark off the attendance of all the children including those that arrived the day before.
  4. A medical specialist using a form called “vaccinate”, presses refresh in fieldTask before checking the children in a class.  This will load the latest vaccination details into their phone.

CSV File

enrol2

Children CSV file

This process is shown in the following video

Automation (Future)

The video showed a back office worker manually copying the daily enrolment data into the children CSV file.  This task could be automated by implementing a custom web page.  Apart from saving some effort for the back office administrators this would also apply updates faster resulting in more current data.

Other Applications of CSV files with Smap

CSV files storing data can be used in other applications, not just for school enrolments and vaccinations.  For example asset management, managing workforce etc.

Version 14.09 – Form Refresh and Export to Neo4j

Leave a reply

Version 14.09 of Smap is available for download.

Refresh – Synchronising phones with the server

The refresh button on fieldTask now does more than just clear the submitted results from the screen. It also now refreshes the forms that you have on your phone and submits any finalised results.  Note you will need to download fieldTask version 2.3.5 from your server on to your phone to use this new feature.

Pressing refresh will now:

        • Download new forms from the server that have been assigned to the user of the phone
        • Show available forms with a yellow clipboard icon
        • Update forms on the phone that have a new version
        • Delete forms from the phone for which the user no longer has access
        • Submit finalised results from the phone to the server
        • Notify the server of the list of forms and their version that the user has downloaded to their phone

You can select a form from the task list screen to complete a survey.  The name, version and project containing the form are shown next to the icon.

device-2014-09-20-090412

Phone with two forms

This new feature should simplify the training of the data collectors as the steps they need to download new forms, start a survey and submit results are all simpler.

Checking that data collectors have the correct version of a form

Sometimes it can happen that data collectors go out into the field with the wrong version of a form on  their phone.  At worst this may have an impact on quality of the collected data and at best you will need to do some messy combining of the data from the two forms prior to analysis. This new feature makes it much easier to check that all the data collectors have the correct version of the form.

From the monitor page select forms.  This will show the users, phones, downloaded forms and the versions of those forms that have been downloaded.  Any users / phones that have not downloaded a form or have an older version are highlighted.

monitor

Monitor Downloaded Forms

 Exporting results to Neo4J / Thingsat

You can now export results to the Neo4J graph database. This change was developed in collaboration with thingsat.  The following video shows how this can be done.

Other Changes

  1. You can edit collected data using web forms.  This capability is enabled by an administrator for your organisation.  By default it is off.  If you select a table showing data for the whole of survey then there will be a button next to each data record that will open the survey results in a web form. If the data is updated the old results will be marked as bad, and then the updated results added.
  2. When you attempt to submit results from a phone for a survey that has been deleted on the server then the error message will be “Not Found” rather than “Not Authorised”
  3. The error message shown, if you upload a survey template that has duplicate options, has been made clearer

Forward Results to another Server

Leave a reply

Beginning in Smap version 14.03 you can add forwarding to a survey template.  This will result in each submitted result being forwarded on to the target server.

Key characteristics

  • Each forward link sends results from a single survey template to a single remote server
  • You can add multiple forward links to a template allowing you to forward results to many remote servers
  • Submissions are sent in one direction only. Hence if the target server is receiving results from multiple sources it will have a different view of the results than each of the sources
  • Forwarding links can be added to the target server as well. These can point back to the original source server but it ahs to be to a different survey on that server
  • Existing results will be forwarded if you add forwarding to a template that already has submitted results
  • Results will queue until there is a network connection before being forwarded
  • The target server can be another Smap server or any server that supports Java Rosa such as ODK Aggregate

Offline laptops

One way of using this forwarding feature is to install Smap on a laptop.  This laptop can be used to aggregate and analyse data collected on mobile phones in areas that do not have a reliable Internet connection.  You can upload the results from the phones to the local laptop over a wi-fi hotspot.  Once you connect the laptop to the Internet it can forward the results onto a cloud server so that they can be worked on by a distributed team.

Community Laptops

image (6)

Community laptops are an extension of the “offline laptop” concept.  Each community can maintain its local view of data collected on the phones.  However they can set up a forward link so that a consolidated view of all the data from all the communities can be aggregated.

Demonstration

The following video shows the forwarding of results from a Smap Server to an ODK Aggreate server.

How to add a forward link

  1. Create the same survey template on the local and remote server.  These templates can have different names on each server but should be the same template.  A good way to do this is to download the template as an XML file from one of the servers and then upload it to the other.
  2. On the source server, find your template on the template management screen and click the upward pointing arrow next to its name
  3. Click the button “Add Forward”
  4. Enter the url of the destination server
  5. Enter a user ID and password.  You should use a user id that only has privileges to complete surveys (enum) as the password is stored in the database. So don’t use your Admin user id!
  6. Press the button to refresh the list of available surveys on the remote server
  7. Select the survey that you want to update
  8. Save

You can monitor forwarded results on the monitor page by selecting the “forward” radio button.

Release of fieldTask v3.0

Leave a reply

Announcing fieldTask version 3.0. This release includes the changes in the odkCollect 1.4 release (thanks to the ODK team) along with the ability to send partially completed forms back to the server. You can download version 3 from google play and from the smap development server. It will be deployed onto other servers as they get upgraded.

ft3

fieldTask v3.0 Home Screen

1) Change Location. [Provided by ODK 1.4] There is a new ‘placement-map’ appearance for location questions.  If you use this “appearance” setting on a question of type “geopoint” then your current location will be shown on a map and you can do a long press on the map to change the position that will be recorded. The meaning of the 4 icons on the map is:

        1. Exit map without saving location
        2. Save location and exit map
        3. Place marker at current location
        4. Zoom to marker

location placement

2) New functions for date processing. [Provided by ODK1.4] Refer to the survey editing guideline and the example xls template.

3) Improved styling for phones with Android 3+. These will show an action bar with icons for some of the menu options. On smaller phones only one or two of the icons will be shown, the other menu options are accessible though the menu button.

icons

Android 2.2 phone menu showing all the icons. Android 3+ phones will show some icons on title bar, remaining menu options will be text when accessed through menu.

4) Access to question responses within repeating groups. [Provided by ODK 1.4].  This allows you to create one repeating group to get all the names of people in a household, then in a second repeating group you can ask detailed questions about each member of the household and refer to the name recorded for that householder. Refer to survey editing guideline and the example xls template.

5) Ability to defer constraint and “required” validation to the end of the survey. [Provided by ODK 1.4]. This can be enabled in the settings.

6) Sending partially completed forms back to the server.  If you have to save a survey half way through because it cannot be completed you can optionally send those results to the server.  After selecting “submit finished data” from the menu, press the menu button again and select “change view”. This will have two options “Show Complete Forms” and “Show Complete and Incomplete Forms”.  If you select the second option you will be able to see the saved incomplete forms and send them back to the server.

There will be an update to the server released by the 18th of October that will mark these incomplete surveys as “bad” with a reason of “incomplete”.  Hence they will not be included in exported spreadsheets or graphs unless the analyst marks them as “good”. If it is critical for you not to allow incomplete forms to be sent until the server is updated to mark them as bad by default, then defer use of fieldTask 3.0 until after the 18th of October.

Testing a long survey

Leave a reply

When creating long surveys for fieldTask you may want to try out a newly created, or modified section, that is a long way from the start. Its time consuming having to swipe past a lot of early questions and even worse if these have been set as “required”.  To quickly get to the right section you can use the “Go To Prompt” feature. Figure 1.

Start the survey in fieldTask or, odkCollect, select “menu” then “Go To Prompt”. A list of questions will be displayed which you can scroll through to find the question you want.

If you have repeating groups in your survey then this list of questions will initially stop at the repeat group.  To go further press on the downward arrow at the left of the repeat group name. Figure 2.

 

end2

Figure 3

Then press on the name that will appear at the bottom of the screen. The questions in the repeating group will be shown and you can continue to scroll down to find the section that you want to test.

Exporting Surveys to Openstreetmap

Leave a reply

I have added a new option to the Smap Server that will export a survey in OSM format. The following video shows this being used to add house numbers to Openstreetmap.

The survey from that video is defined in an XLSForm template reproduced below:
osmsurvey

This survey will collect the street name, odd or even, country, city, postcode in the top level form. It then collects the house numbers and their locations in a repeating group.  Some of the top level fields are copied into each house record using the “calculate” type.

I have adopted a couple of naming conventions for surveys that are destined to be exported to osm.

  1. Any question name that starts with an underscore, such as _country, is not exported.
  2. If the question name starts with “addr_”, then that is changed to “addr:”. The reason for this is that XLSForm did not seem to support the use of colons inside names but they are used in OSM key values.

I completed the survey twice to get the house numbers for a street called Jewell Crescent. When the results are exported the OSM file looks like this:

[code language=”xml”]

























































[/code]

After loading into Openstreetmap it looks like this.

This new functionality is in the source code now. It will be included in the next production release of Smap on Sunday 28th of July, after which the server can be downloaded from here.
Other Thoughts

  1. Currently option names map to OSM values.  This is quite restrictive as ODK does not allow these names to have spaces.  I will probably add an option to the export function that, if set, will use the option label to set the OSM value.
  2. You can export any survey to Openstreetmap.  Most of the time these won’t have been designed to be loaded directly as the demo was.  However it can still be useful to get the data into JOSM and then manually copy information into new OSM nodes.  For example if you have a survey with information on health clinics.

Connected Arabic fonts in Android 2.3

3 Replies

I’m working with a customer that is conducting a survey in Iraq using Samsung Galaxy Y Duos (android 2.3) and fieldTask. Later model phones (android 3+) worked well with Arabic but not these earlier versions. Even after rooting the phones and installing new fonts it still didn’t connect the characters.  See below a screen shot using Dr Ben Ox Arabic fonts which seemed to give the best results.
arabic2 (1)

Fortunately there is an Arabic reshaping library available from https://github.com/agawish/Better-Arabic-Reshaper/ which can be used with these old phones. I added ArabicReshaper.java and ArabicUtilities.java from this library to the odkCollect library that performs survey completion in fieldTask. I then modified the “QuestionWidget.java”, “SelectOneWidget.java” and “SelectMultipleWidget.java” files in odkCollect to call the “reshape” function when setting the text. See line 12 below.

[code language=”java” highlight=”11,12″]
protected void addQuestionText(FormEntryPrompt p) {
String imageURI = p.getImageText();
String audioURI = p.getAudioText();
String videoURI = p.getSpecialFormQuestionText(“video”);

// shown when image is clicked
String bigImageURI = p.getSpecialFormQuestionText(“big-image”);

// Add the text view. Textview always exists, regardless of whether there’s text.
mQuestionText = new TextView(getContext());
// mQuestionText.setText(p.getLongText()); smap
mQuestionText.setText(ArabicUtilities.reshape(p.getLongText())); // smap
mQuestionText.setTextSize(TypedValue.COMPLEX_UNIT_DIP, mQuestionFontsize);
mQuestionText.setTypeface(null, Typeface.BOLD);
mQuestionText.setPadding(0, 0, 0, 7);
mQuestionText.setId(QuestionWidget.newUniqueId()); // assign random id

// Wrap to the size of the parent view
mQuestionText.setHorizontallyScrolling(false);

if (p.getLongText() == null) {
mQuestionText.setVisibility(GONE);
}

// Create the layout for audio, image, text
mediaLayout = new MediaLayout(getContext());
mediaLayout.setAVT(p.getIndex(), “”, mQuestionText, audioURI, imageURI, videoURI, bigImageURI);

addView(mediaLayout, mLayout);
}
[/code]

This improved the display significantly:
SC20130708-172712

This Arabic version of fieldTask is available for download from http://dev/smap.com.au. There are other widgets that could have this functionality added and presumably the arabic text typed by an enumerator in a text field should also be reshaped. These additional changes will be made on an as needs basis. The preferred option is to get a phone that supports Arabic fonts natively which would be a better option than having to root the phones to install new fonts.