Tasks
How do we re-assign tasks to a new data collector?
- Go to the Task Module
- Select the task group in question
- If you can’ see the tasks that you want to re-assign then change the period to “All” (By default only tasks assigned in the last week are shown)
- Click on the edit button next to a task that is assigned and that you want to reassign.
- Select the user you now want to complete that task and press save
We know that one data collector has cooked the data how do we remove this data from the reports?
- Goto the Analysis Module
- Click on table view and select the survey
- Enter into the advanced filter:
- ${_user} = ‘data collector ident’
- For example: ${_user} = ‘tom’
- In the advanced filter you can refer to questions using the $(question_name} syntax. In this case we are using the special name _user to refer to the login id of the user who submitted the data
- This will show the records submitted by the user you specified
- Then you can inspect the data and if it is bad right click with your mouse on the green cell with the heading “Marked Bad”. You can enter a reason if you want. This will set the colour to red. These bad records will not be included in exports or graphs. The only place you can see them is in a table view.
- If you want to toggle it back to good again you just right click with your mouse on the red cell.
Having fixed up the “cooked” data how do we get another user to go and redo the interview?
- Go to the Tasks Module
- Select a task group of interest
- Select the tasks that you want to redo. This is done by selecting the checkbox to the left of the task
- Select the menu option “Tasks” then “Set selected as assigned”.
- The tasks will now be ready to be done again. You will then presumably need to assign a different user to them as described above.
Analysis
How do I fix errors in the collected data?
There are a number of ways to do this and a tutorial has been created to answer this question. You can find it here.
Administration
How do I get an easy to read version of my survey in a PDF?
In the template management screen click on the blue downward pointing arrow next to the survey you want. Select PDF, then select the language that you want. Click download.
Do I need to convert my spreadsheet template to XML before loading it onto the Smap server?
No you can load an XLS or XLSX survey directly into Smap. The spreadsheet name can now also have spaces in it.
I added a new project but when I select “projects” in template management it does not appear in the list?
Users that create a new project are not automatically given access to that project. After you have created your project click on your name and the names of other people who need access and add the new project.
Update: If you create a new project you will be automatically added as a user of that project.
Trouble Shooting
I can’t see results from submissions in the database
There are a number of possible reasons for this and you can usually find the answer on the monitoring page. This hint covers a particular instance of this problem and describes how to access the monitoring page and resolve the issue when it is caused by a question type being changed after data collection is started.
The server is not responding when I try to look at results or go to administration!
If you have a working network connection then the most common cause of this problem is after a new release has updated the application on the server. Your browser will have stored Smap web pages in its cache and it is possible that these old cached pages are no longer compatible with the new server. application Try pressing “Ctrl – F5” on Windows. “Command – r” on a Mac, this will refresh your browser’s cache. More detailed instructions are available here: http://www.refreshyourcache.com/en/home/
If the above quick refresh fails try clearing your whole cache. In Chrome select the “Chrome” menu option and then “Clear Browsing Data”. In Firefox select “History” then “Clear recent history”. Remove cookies, cache and active logins.
Some features on the web server, such as graphs, don’t seem to be working!
Smap is only tested on Chrome and Firefox. If you are using Internet Explorer try with one of these supported browsers.
Arabic text in my survey does not display well and the characters are not connected!
Older versions of Android do not support Arabic well. Survey teams have had good results with Android 4.1 phones both for the display of Arabic text and entry of text. If you only need to display Arabic but not do text entry then there is a special Arabic version of fieldTask that will work on Android 2.2+. This is available for download from http://dev.smap.com.au This download is no longer available, it worked fine for displaying Arabic but you could not enter Arabic text. The solution is to use Android 4.1+.
When I try to start a survey in fieldTask on my phone, I see a message “Unfortunately, ODK Collect has stopped”.
You should not see this error anymore as the fieldTask implementation of forms processing is isolated from the implementation by odkCollect so you should be able to use both tools side by side.
fieldTask uses software from odk to complete surveys. This software, which is included in fieldTask, has been modified a little. If you have been running odkCollect on your phone Android will have been set to automatically run the odkCollect version of this code causing fieldTask to stop. This can happen in reverse as well, when you try and run a survey from odkCollect it will fail if you have been previously running fieldTask.
The easiest fix is to only have either fieldTask or odkCollect on your phone at one time. You can un-install one of them by going to “menu” then “settings” then “Apps”. Alternatively select the survey tool that you are not using from the “Apps” menu. Then select “Clear Defaults”. The next time you start a survey you will be asked which survey tool you want to use.
Neil, how does fieldTask work for off the grid (no Internet access) data collection?
FieldTask is an Android application that can collect data without any internet access. The normal approach, and most straightforward, is to:
1) Download surveys onto the phone (over wi-fi, or a cellular data connection)
2) Head out into the field and complete multiple surveys without any Internet
3) Get back to base and upload all the results to the server over wi-fi or the cellular network.
However you can complete all stages without any Internet connection. You can load surveys onto the phone using USB cable and also copy results off the phone onto a PC via the same mechanism. Your PC would then only need to be connected to the network in order to upload the data to a server.
Another approach that some of our clients are using in Brazil and Guyana is to put the server onto a laptop. They are using a local wi-fi network to copy data to and from the phones (easier than using cables) but the laptop is not connected to the internet. They can then analyse the data offline.
Webforms, the browser version of fieldTask also works offline using HTML5 caching.
regards
Neil
Hello Neil,
How to register on the SMAP server?
Hi Koassi,
Thats a good question. I am planning on making a server available for people to use along the lines of formHub. People would just need to provide an email address as a unique form of identification and the server would create an “organisation” and add them as an administrator. An organisation applies a logical separation of data within the database. However this public server is not available yet.
There are 3 current options I think:
1. Setup your own Ubuntu server and download and install Smap server onto that. Details at http://www.smap.com.au/downloads.shtml
2. I can create an organisation for you on my demo server. The url would be http://demo.smap.com.au. Actually this would be similar to the idea of making a public server available except the setup is manual. Its quite stable and you could do production assessments using it. There would be no support available and no guarantees other than my naturally helpful nature. However the database is backed up nightly and would be restored in the event of a catastrophic crash.
3. I can set up a hosted server just for you on AWS. That would be $100 per month and comes with IT support and help desk
regards
Neil
Dear Neil,
We are currently using a VPS to host our ODK Aggregate, is it possible to use SMAP instead of the VPS to reduce the charges?
Regards,
Ronald.
Hi Ronald,
There is a free hosted Smap server that you can use but only for work with not for profit organisations. You can register at https://sg.smap.com.au. If you have commercial clients send me an email at neilpenman@gmail.com and I can provide a quote.
regards
Neil
Hello Neil,
I evaluated ODK Aggregate & Formhub / Ona, but SMAP seems to be different! Really great,
this is what I could see from features page from the Website. I could not evaluate SMAP like above two, but got overall idea. Still have a few questions about SMAP server.
Can I use ODK Briefcase to download data from SMAP?
How it deals with select_multiple fields? Are these stored as space separated values in a cell? (for viewing online, downloaded as CSV from server & downloaded with ODK Briefcase) There are different results from different servers. Does it make different column for each answer ?
Whether media are stored as blob field in database or in a filesystem folder like media, attachments etc?
How time is stored? Actual form upload time or as GMT+5:30, in my case?
The pivot export feature is just great & very much needed.
Overall, I am very much impressed by this fascinating software. & hence want to know more about it.
Request to answer some of the few questions .
Thank you, with Regards.
Hi Abhay,
Thank you for looking at Smap and your kind comments. Installation of Smap Server on an Ubuntu machine is automated and pretty straight forward. I would also be happy to give you an account on my demo machine so you can do some more in-depth evaluation.
Here are my answers to your question:
1. Briefcase is not supported with Smap. I don’t think its really needed for the following reasons, and I’d be interested to hear your opinion on this.
a) You can combine forms and subforms into a single table on export.
b) Each choice from a multiple select question is stored in its own database column and exported in its own spreadsheet column.
c) There is a simplified version of Briefcase called “Smap Uploader” which handles uploading survey results from disk.
d) I don’t support encrypted surveys. You can encrypt on the device with Android tools, the network with SSl and the database backups with PGP. You can even encrypt the server disk but since its running all the time thats of limited value. So my approach is then to focus the security effort in making the server secure.
2. Select multiples are stored one column per choice with a value of “0” or “1”. So when they are exported the idea is that you can easily count the number of selections for each choice.
3. Media files are stored as files on the server file system. This is the case with both media collected during a survey and media that is used for labels. An Apache HTTP server, which front-ends Tomcat, serves up the media files directly.
4. Times are stored with the time zone in the Postgres / Postgis database. When exporting the survey results, times are in GMT time zone. In excel you can then insert a new column and set the value (in your case) to date_col + 5.5/24 which will then display in your time zone.
It would be great to collaborate further.
Regards
Neil
Your comment is awaiting moderation.
I installed the last version. I uploaded by app and with form data, but smap server not create the tables to storage the information and in monitoring only can see in submitted (instances) the xml and appears: Not added to database Why? ws user is a superuser.
We Plan Bangladesh using fieldstadk but i am in problem. my existing form titled school mapping need to be edit. How can i edit my existing form titled school mapping.
Hi Shovan. It depends on what your objectives are. The best way to edit your existing form is to use the excel editor. When you load the new form it will get a new name and the data you collect with it will be spearate from the existing form’s data. If you want to edit the existing form and combine the data from the modified form with the new form then you will need to use the online editor. However the functionality in the online editor is limited and its very much beta code at the moment.
Hello Neil, thank you very much for you reply.
If i wanted to edit the existing form and combine the data from the modified form with the new form then i have to use the online editor so far I have understood from your reply. And you also mentioned in your reply, functionality in the online editor is limited and its very much beta code at the moment.
So, as I understood its not easy task, right and I am not allow to do edit the existing form.
So, what is the best way to fix the problem?
Thank you very much.
Hi Shovan,
You can combine the data from the two surveys outside of Smap in a spreadsheet. So if your main analysis is done there and you don’t need to combine the data too many times then it will be very simple to line up the headings from each survey and merge them.
With the online editor I am close to releasing a production ready version. Hopefully as soon as this coming weekend although its a large, complex job so it could be delayed. It will still be Beta software so I’d recommend you use the spreadsheet editor where you can until the next release in January. However for editing an existing survey it should work fine.
What exactly do you want to change in your existing survey. I can try and make sure that is supported in this weekend’s (hopefully) coming release.
Hi Neil, one of my users accidentally used an old version of a survey. The original template has since been deleted on the server and a new one (with the same name) uploaded; he forgot to refresh the forms before going out into the field for collection. If I tap the survey in the phone nothing happens; if I tap and hold, it just says “Status complete”. I can’t submit finished data (given the forms don’t exactly match, there is one extra option on one question, I get the Error: Not Found (404)). How can I view the old surveys he’s completed with participants (which seem to be “stuck” on the phone), so I can manually extract the data?
Hi Elise,
Was the form just “deleted” or “erased”? There are 2 stages to deleting a form. The first by pressing the delete button just makes it invisible. However the form, and its data are still there. You would be able to restore it by:
1. On the template management page click the checkbox that says “Show deleted”
2. Find the form that your user downloaded, it will have the date it was deleted appended to its name,
3. select it and then click “restore” at the bottom of the page
You should then be able to upload the data.
Alternatively if the form was completely erased we will still be able to get it back but it will be trickier.
Are you on a hosted server? If so I can have a look for you. But try the “restore” first.
You can ask support questions by emailing smap-suite@googlegroups.com. That may be easier than posting to the blog.
regards
Neil
For the record we addressed this issue offline. The 404 error, ‘Not Found’ is shown on the phone if a survey has been deleted on the server. You can un-delete the survey and then resend from the phone in order to get the data.
Thanks Neil,
All our work at the moment is for Not for Profit organisations. We are greatful to use the free hosted server. However, will the free hosted Smap server allow for creation of two or more projects at ago by a single user?
Regards,
Ronald.
There are no restrictions apart from the commercial use. Once you have registered your organization you can create multiple users, projects, surveys and collect as many assessments as you like.
Dear Neil
I have created the form . There is multi language support in smap.
But i am not able to use hindi/nepali language.
whats the process to use hindi/nepali language
Hi Suprim,
Within the form itself you should be able to add Nepali / Hindi directly under the label column. As long as your phone supports Nepali the questions should appear in that language.
Currently I don’t think fieldTask supports Nepali although it should do so after the next upgrade. Hence the buttons in the phone application may not appear in Nepali.
The server is not currently available in Nepali although it does have some translations for Hindi. You are welcome to contribute to those translations on transifex.
Hi,
After installing on ubuntu 12.04, I cannot start apache2.
The error reads, “Syntax error on line 43 of /etc/apache2/sites-available/smap-volatile.conf:
Invalid command ‘AuthBasicUseDigestAlgorithm’, perhaps misspelled or defined by a module not included in the server configuration”
I have tried re-installing apache2, but its still not working.
Any possible solutions to my problem?
Hi,
Smap requires Apache 2.4 now. I would recommend you install 14.04 as that will have Apache 2.4 and is currently supported by canonical.
The latest LTS released is 16.04. Unfortunately i havent had the chance to test Smap on that. Probably there will need io be changes to the install script if you go with that one.
Hi Neil,
I managed to install SMAP and created form which has geo location activity. It’s take to much time to get Geo Location i have to manually save coordinate. When I tried to send the data no error occurred but on the form result the geo location was empy. Do you have any idea why that happen?
Thank you,
Hi Edwin. Did you run this form on a phone or in a web browser?
Web browsers generally ask for permission to record location. If its on a phone then is gps coordinates working for other apps?
On the phone the coordinates will be recorded when the accuracy is better than 5m. However you can manually override that if for example you are inside a building.
This all looks very good. I currently am using ODK aggregate and Enketo web forms or ODK-collect, but need the features that include bi-directional synch and the ability to complete surveys in multiple stages…partially comlpete, update, the same form..etc.
And I am publishing our results to Google Fusion Tables.
Is there any way to ‘publish’ the results of the survey to google sheets or fusion tables like the ODK Aggregate does?
Also, is there an ios version of the fieldTask app?
Hi Lloyd, No there is no “publish” to google fusion at the moment. You would have to export as excel and then import manually. There is an ios version of fieldTask called zarkmanMobile.
Thanks Neil,
Do you know if I have my own smap server setup, with the ZarkmanMobile app will work with it? or does it only work with Zarkman server or ‘hosted’ smap servers?
Ah of course my apologies I should have said. You are correct. No Zarkman Mobile is not open source. I’m allowed to make it available through servers I host but apart from that it only works with servers provided by Zarkman who cater for commercial customers. Webforms works pretty well on ios even off line. I will be shortly making a new release which fixes some issues with ios 10.2.
Hello
i make the register to the Public Shared Server but i don´t receive the email confirmation, What i should do?
Hi Neil, please I want to find if you have any step by step videos online on how to install smap server on ubuntu. Thanks
Hi,
No there are no videos on that. However the installation process is straightforward you can find details in the operations manaual that is available for download from the same web page as the server installation file.
regards
Neil
Using the public server can I administer 1,000 questionnaires including pictures of respondents. How long will responses uploaded on the public server be kept and made available for use
Hi,
Sorry for the delay in replying, missed your post.
The short answer is there is no fixed limit to the number of questionnaires you can complete. Responses will be retained on the server indefinitely or until you delete the survey.
However as you may notice in the terms and conditions if I become aware of what I would consider excessive usage then I may add restrictions to the account. The numbers you are talking about however are fine.
regards
Neil
Is is possible to change / modify the data after sending from ODK app? On the SMAP Server?
For example, i created a survey with one select list with animals(cat, dog, bird), after take some records and send them to server i need to add new item on the list. I add (+ mouse) but on the before data i need to change some data value / result . Is it possible?
Hi Jose,
Yes it is possible. For a select one type question however you will need to open with the web form editor. You can then change any data in the record that you like. Note this capability is not enabled by default for an organisation. Let me know if you want me to enable it. Once enabled the approach is:
There is also a “Modify” feature in the analysis module. This can be used with text questions to very quickly make bulk changes such as making spelling consistent. It can also be used to convert “Other” text responses into a select one attribute. However it can’t be used to modify select question values.
Regards
Neil
Just for completeness there are a couple of other ways to update collected data.
Firstly using managed forms. This feature allows you to attach a second “oversight” form to each of your forms. This second form can have questions such as a comment or a status. Using the managed forms interface in the webforms module you can then set a status or comment for each of the records collected using your primary form. If you add a question in the oversight form that has the same name as a question in the primary form then any changes you make will update the original data. So in your case this question would be the select one with additional choices. Detailed help is available here: https://drive.google.com/open?id=0B5_SmpWlQYxvRHZ6Z0ZYMGNJS3M
Secondly you can export all your data. Modify it in excel and then re-import. Export / import is also done from the analysis module. Again create a table panel then select the side draw by clicking on the “<" symbol. You will then see the export and import buttons.
Hello,
Whenever i develop a form with multiple select options coming from a csv, my sent data doesnt show on the web. Could someone help with that.
You can send an email to smap-suite@googlegroups.com and attach your form if you think thats relevant to the question. You will ned to provide some more detail on what you mean by “sent data doesn’t show on the web”.
hello Neil Penman,
I meant that if am having options for a multiple select coming from a csv, I will never see my Data sent from the phone using fieldtask
on the web report/server
Hi Neil Penman,
I am interested to use smap with my own server. It seems to have great features other than ODK-based solutions.
I see currently you are in 17.09 version, but more recent for download (https://www.smap.com.au/downloads.shtml) is 17.06.19. When do the latest version could be available to download or at least 17.08?
Hi Diego,
A good question! A lot of the new functionality that I am currently adding to Smap uses AWS services at the backend. This has resulted in a delay in the release of a downloadable version. The considerations are:
1. I need to make sure AWS config files specific to Smap are not included
2. AWS service calls should be seemlessly avoided if the user has downloaded the application
3. I should probably provide a way for an installer to enable these AWS calls to their own AWS account
Because I haven’t put much thought into the above I haven’t released an update recently. However I will try and do this relatively soon to at least address 1, and 2 above.
You should also be aware that a user reported issues installing the software on Ubuntu 16.04. The problem was related to the version of Postgres that got installed. I think this was because they were using the desktop version of Ubuntu as when I tested the server version I was still seeing Postgres 9.5 which the install script assumes. If you come across this issue and manage to fix the install script to cater for this situation then that contribution will be gratefully accepted.
Hi Neil
Is it feasible to regularly update a CSV file (say every hour), and if so, can it be automated in the smap server? I would like fieldworkers to have a very recent list of unique person identifiers (captured in an enrollment form), so that when capturing services for a person, the CSV will insert the correct unique person identifier into the service form.
Hi Rob,
Yes you can do this. Firstly in offline mode where csv files which can be copied automatically to the phone when the survey from which they derive changes. Simply reference the survey in your pulldata function like this: pulldata(‘linked_s_{suvey ident}’,…). Secondly you can lookup data in realtime mode when online using the lookup function lookup(‘linked_s_{survey_ident}’…). This second approach has the advantage that if you have a very large number of records in the reference survey you do not need to copy them down to every phone every time the data changes.
Hi Neil
I need your support
lacro.smap.com. au say
Error: Service Unavailable
Hello Neil,
Neil could you give me certified me of ODK as you facilitated training me 17-Nov-2016 to 18-Nov-2016 because I am giving the training to social science student.
Neil can you create a cluster for me on your server for that training.
Could you email me at sushil.singh@herd.org.np
Regards,
Sushil Singh
ICT officer
HERD Nepal
Hi Neil, is there any limit to number of forms in a survey? Or is there any limitation for the size of the survey data file (in MB)?
Hi Bibin. There are no prescribed limits. If you have a lot of images in your results or large videos then you can get timeouts while uploading particularly over a slow network.
Dear Neil,
I am sorry I don’t know if the question is posted at its due place. First, I would like to express many thinks to you for this tool (SMAP) I was impressed with its features and want to adopt it. However I have a problem to get what I want to do: I am preparing a household survey where I want to list first all the household members and according to some eligibility criterias selected questions whithin a unique questionnaire are asked. This means I will have to list members first and after this come back to some (not all) for additional questions. I have to mention that for each HH member, there are some informtion collected. So is there a way around to get reconnect to a HH member for more questions and return back to another member?
Hi Menus,
I’m glad you find the system useful. I think you can do what you are asking for using the indexed-repeat() and position() functions to reference questions in one repeating group in another repeating group. You can then use relevance to only ask questions in the second group that meet your criteria.
The following is a link to an example survey that does this. In the first loop you enter name and age, In the second loop it will ask the gender of all householders that are older than 10.
https://drive.google.com/open?id=1eRBr–j_bFxyAQZFotNzRg0Z_KzjKodK
Yes! Thank you very much I got it!
Dear Neil,
I have a form which is half filled and saved. When I opened it for refilling new data, the keyboard is unavailable. Is there a parmeter to control for this issue in FieldTask or it was a android problem independant of smap and FiledTask?
Hi Menus,
This does not sound normal. I just did a quick test:
The form opened on the text question with the keyboard visible. If you are still having problems can you email me at neilpenman@gmail.com with more details. FieldTask version number, a screen shot, perhaps attach the form.
Hi Neil,
I got the way. When using the back button and save changes, the keyboard is available (the saved form icon is blue) BUT when using Save Form and Exit, the keyboard in unavailable and then text or numeric field can not be modified (the saved form icon is purple).
Is it expected like that?
I prefered to write here, it may help others.
Ah yes that is expected. When you get to the end of the survey and press the “Save Form and Exit” button then the survey is saved as “finalized”. It can no longer be edited and is just waiting to be sent to the server. In this state it has the purple icon.
The only thing you can change are the comments. Hence you can open the survey and find questions where you want to add a comment then select menu then comment.
So if you want to continue working on a survey you need to press the back button and save the changes.
Hi, I want to know if there is a way for enumerators to see tasks that are unassigned, but allowed by
projects/roles/groups , so they can assign to themselves or accept a task by themselves?
Thanks in advance
Hi Alan,
No tasks are assigned to users by an administrator or they are assigned automatically according to the rules for the task group.
What you are suggesting is however exactly how the console works. Each record of work in the console can be “locked” by a user at which point they are working in it and they can update the data and create tasks for other people to update the data. The user can filter by records that are outstanding and are not locked by another user, ie unassigned.
It would be an enhancement to allow tasks to be treated the same way. Seems reasonable. Perhaps a setting in the task group that tasks allows self acceptance by users.
Hi, that’s great!
so do you think that this improvement is hard to implement?
I think that will be amazing for users to see into the fieldtask map what’s is available to do close to then and pick their own tasks without intervention of somebody else.
Also, I wonder if there is a way to ‘auto-create’ tasks, like triggers do inside databases, because it can improve things without some admin’s hands too =)
Do you have some model to do consulting to a company that hosts his own smap server offline? I’ve read about some guys doing it using notebooks as servers, and I am considering doing something quite alike.
By the way, very nice software suite, smap server/fieldtask, and open source! Amazing work! =D
Looks quite straightforward. I have created an enhancement issue to manage its implementation. https://github.com/smap-consulting/smapserver/issues/538
Auto creation of tasks is currently only done on a records submission. I would be interested in hearing your views on other occasions when this should happen. Email me to discuss. neilpenman@gmail.com
In terms of your consulting if you work in international development then just let me know and I can send you the latest server implementation to download. If you work in corporate or government then email me. I would be delighted to work with you.
Looks promising, thanks for that, I think all users will like this feature when implemented. =)
I would ask about the latest Version to make some hardware performance testing, so I shall email you right away, thanks!
Best regards, and let’s move on! =D
What is the meaning of the word SMAP :)?
When we first started in 2007 our objective was to put open source maps onto phones in order to create digital infrastructure for development. Hence StreetMAP. However we were diverted into more general data collection with phones. As you know the vestiges of that original approach are still there in aspects such as the geospatial database.
Hi Neil
Is there a way to connect the server to a database?
Hi Sukaina,
The server works with a postgresql database. If you use postgres yourself then you can point the app server at your own database or do table to table replication. If you use a different database type then you can use ETL tools or your own batch processes to sync the databases.
Hello Neil,
I would like to know if the platform can connect with Microsoft Dynamics 365 (CRM). I would like to know if we would need something like Zapier or it intermediary is not necessary.
Rhode
Hi Rhode,
Yes I think a middleware tool like Zapier would be your best bet. The most common type of integration is with dashboards like Power BI where you can specify a data API to get the results. I’m not sure if Microsoft Dynamics allows that.
Regards
Neil
Hello
Mistakenly i have remove admin from my ID how can assign me as admin again?
Hi Punam,
My apologies for the slow response, I don’t monitor this channel closely. You will need to get another administrator to add you back in. If there is no other administrator and you own the server then you can add a new user by updating the database directly. If you are using the SG public server then I can do that for you.
Regards
Neil