Kobotoolbox is one of the most widely used to for mobile data collection. It is used by many humanitarian organizations to collect data online, offline and in challenging settings.

Once collected, you can export the data in Excel format through the platform to run your analysis. However, as data is being collected, manually exporting files for analysis is not the best way to build a robust pipeline for automatic and reproducible data analysis.

One approach to fix this problem is to build set of tools to streamline analysis of Kobotoolbox data through its new API. The use of the API fix one problem which is the automatic access to the data but to also have a reproducible workflow we need a flexible programming languages for data analysis like R.

robotoolbox is a new R package created to tackle these issues. It allows analyst to quickly get their data from Kobotoolbox to R.

Install robotoolbox

robotoolbox is still in development and not yet on CRAN. In order to install it, you will need the remotes package.

## install.packages("remotes")
remotes::install_github("dickoa/robotoolbox")

robotoolbox is built around KoBoToolbox API v2 and its main goal is to ease the process by which you can access your collected data.

In order to setup robotoolbox, you will need to set your API token and specify the KoboToolbox server URL you use. The easiest way to setup robotoolbox is to store the token and url your .Renviron.

We are using the following environment variables KOBOTOOLBOX_URL and KOBOTOOLBOX_TOKEN to respectively store the server url and token. You can use the usethis R package and the usethis::edit_r_environ() function to add these environment variables to your .Renviron file.

KOBOTOOLBOX_URL="https://kobo.unhcr.org/"
KOBOTOOLBOX_TOKEN=xxxxxxxxxxxxxxxxxxxxxxxxxx

In an interactive session, you can use directly the kobo_setup function.

library("robotoolbox")
kobo_setup(url = "https://kobo.unhcr.org",
           token = "xxxxxxxxxxxxxxxxxxxxxxxxxx")

You can get your token manually through the KoBotoolbox web interface in your account settings. There is also the kobo_token function to do the same thing automagically from R.

token <- kobo_token(username = "cool_user_name",
                    password = "gRe@TP@$$WoRd",
                    overwrite = TRUE)

This token can then be used in the setup.

kobo_setup(url = "https://kobo.unhcr.org",
           token = token)

and it is possible to check the settings with the kobo_settings function.

kobo_settings()
## <robotoolbox settings>
##    KoBoToolbox URL: https://kobo.unhcr.org/
##    KoBoToolbox API Token: xxxxxxxxxxxxxxxxxxxxxxxxxx

Kobotoolbox projects

List your projects

All your Kobotoolbox projects also named assets can be checked and listed through the web interface.

Using robotoolbox, you can do something similar from R with the kobo_asset_list function. The result of a call to kobo_asset_list is a tibble and can be manipulated as such.

library("robotoolbox")
library("dplyr")
al <- kobo_asset_list()

al |>
  slice_head(n = 6)

uid	name	asset_type	owner_username	date_created	date_modified	submissions
a37LegD9joqVEDjaGaDrmP	WCA Programme management capacity building and COMPASS improvement survey	survey	dickoa	2023-01-23 20:02:50	2023-01-27 12:34:52	1
a9bwEQcaWqWaA5hzkjRAWx	UNHCR FORMULAIRE COLLECTE DES INCIDENTS DE PROTECTION - INTERSOS	survey	imrca	2021-03-05 08:04:06	2023-01-04 08:57:57	10876
aA9QwKdEXqfGiQCkPNzKBR	RMS CAPI POUR TEST GISSE	survey	dickoa	2022-12-30 16:23:57	2022-12-30 17:22:02	1
aEZsFfwvADtcAGzuVhd6UL	date_time test	survey	dickoa	2022-12-18 13:31:21	2022-12-18 13:31:42	1
aGHVFT8pFCikCsimq8NtRS	contact data manh test	survey	dickoa	2022-12-18 12:54:02	2022-12-18 12:54:32	1
acoSWjKHdu8nUpFSuAdt2h	2021_UNHCR_CIV_Menage Rapatrie	survey	unhcr_ico	2021-11-10 10:33:14	2022-10-26 09:34:26	12984

KoboToolbox asset

You can also manipulate directly each asset (project) using kobo_asset. An asset is uniquely identified by a unique identifier: the uid.

uid <- "aEwTYNcU76UvLgiY89rPWm"
asset <- kobo_asset(uid)
asset

<robotoolbox asset>  aEwTYNcU76UvLgiY89rPWm 
  Asset name: Pan-African Capacity Building in R
  Asset type: survey
  Asset owner: rrc_rrrp19
  Created: 2022-01-06 14:33:35
  Last modified: 2022-01-28 12:31:28
  Submissions: 95

Since al, the list of assets, is a data.frame, we can subset it and select the uid of interest based on the other available metadata.

As an example, you can get the uid associated to the project with the largest number of submissions.

al |>
  slice_max(submissions) |>
  pull(uid)

[1] "a7XzRuPFn9j5WkT2mR6wbg"

And it’s then possible to get the associated asset object.

al |>
  slice_max(submissions) |>
  pull(uid) |>
  kobo_asset()

<robotoolbox asset>  a7XzRuPFn9j5WkT2mR6wbg 
  Asset name: form for stress test
  Asset type: survey
  Asset owner: tmneche
  Created: 2021-04-14 10:19:19
  Last modified: 2021-05-21 11:31:33
  Submissions: 118261

Kobotoolbox data

The main function on the robotoolbox package is kobo_data (or kobo_submissions its alias) and as the name suggests, it loads your data from the server.

We can illustrate its usage with data from a project named Multiple languages.

The first step, is to get uid of the project (or asset).

uid_ml <- al |>
  filter(name == "Multiple languages") |>
  pull(uid)
uid_ml

[1] "aYuTZn9vegi3Z49MXwKjep"

We can the use uid_ml that uniquely identify the Multiple languages asset to read its data

data_ml <- kobo_data(uid_ml)

data_ml |>
  select(start:uuid)

start	end	today	full_name	pet_yesno	_id	uuid
2022-01-07 10:39:47	2022-01-07 10:40:02	2022-01-07	Fatim	1	17735003	5388c680d19148828ea45913af820f30
2022-01-07 10:39:37	2022-01-07 10:39:47	2022-01-07	Jean-Pierre	0	17735002	5388c680d19148828ea45913af820f30
2022-01-07 10:38:29	2022-01-07 10:39:37	2022-01-07	إبراهيم	1	17735000	5388c680d19148828ea45913af820f30
2022-01-07 10:38:20	2022-01-07 10:38:29	2022-01-07	Michelle	1	17734995	5388c680d19148828ea45913af820f30
2022-01-07 10:37:39	2022-01-07 10:38:20	2022-01-07	Ahmad	0	17734994	5388c680d19148828ea45913af820f30

Kobotoolbox form

You can also check the form used using the kobo_form function

uid_ml |>
  kobo_form()

name	list_name	type	label	lang	kuid	choices
start	NA	start	NA	English (en)	Cypi2O0NQ
end	NA	end	NA	English (en)	F2Erecceg
today	NA	today	NA	English (en)	6WnNQUnlF
full_name	NA	text	What is your name?	English (en)	pkEk0bzX7
full_name	NA	text	Quel est votre nom ?	Francais (fr)	pkEk0bzX7
full_name	NA	text	ما اسمك ؟	Arabic (ar)	pkEk0bzX7
pet_yesno	yesno	select_one	Do you have any pet?	English (en)	a1xXud0rG	1 , 1 , 1 , 0 , 0 , 0 , Yes , Oui , نعم , No , Non , لا , English (en) , Francais (fr), Arabic (ar) , English (en) , Francais (fr), Arabic (ar)
pet_yesno	yesno	select_one	Avez-vous un animal de compagnie ?	Francais (fr)	a1xXud0rG	1 , 1 , 1 , 0 , 0 , 0 , Yes , Oui , نعم , No , Non , لا , English (en) , Francais (fr), Arabic (ar) , English (en) , Francais (fr), Arabic (ar)
pet_yesno	yesno	select_one	هل تمتلك حيوانا أليفا ؟	Arabic (ar)	a1xXud0rG	1 , 1 , 1 , 0 , 0 , 0 , Yes , Oui , نعم , No , Non , لا , English (en) , Francais (fr), Arabic (ar) , English (en) , Francais (fr), Arabic (ar)

Which is a representation in robotoolbox of the following xlsform

Survey questions

type	name	label::English (en)	label::Francais (fr)	label::Arabic (ar)
start	start
end	end
today	today
text	full_name	What is your name?	Quel est votre nom ?	ما اسمك ؟
select_one yesno	pet_yesno	Do you have any pet?	Avez-vous un animal de compagnie ?	هل تمتلك حيوانا أليفا ؟

Choices

list_name	name	label::English (en)	label::Francais (fr)	label::Arabic (ar)
yesno	1	Yes	Oui	نعم
yesno	0	No	Non	لا