robotoolbox: an R client for Kobotoolbox

R
Kobotoolbox
Author

@dickoa">Ahmadou Dicko

Published

January 28, 2023

Kobotoolbox is one of the most widely used to for mobile data collection. It is used by many humanitarian organizations to collect data online, offline and in challenging settings.

Once collected, you can export the data in Excel format through the platform to run your analysis. However, as data is being collected, manually exporting files for analysis is not the best way to build a robust pipeline for automatic and reproducible data analysis.

One approach to fix this problem is to build set of tools to streamline analysis of Kobotoolbox data through its new API. The use of the API fix one problem which is the automatic access to the data but to also have a reproducible workflow we need a flexible programming languages for data analysis like R.

robotoolbox is a new R package created to tackle these issues. It allows analyst to quickly get their data from Kobotoolbox to R.

Install robotoolbox

robotoolbox is still in development and not yet on CRAN. In order to install it, you will need the remotes package.

## install.packages("remotes")
remotes::install_github("dickoa/robotoolbox")

robotoolbox is built around KoBoToolbox API v2 and its main goal is to ease the process by which you can access your collected data.

In order to setup robotoolbox, you will need to set your API token and specify the KoboToolbox server URL you use. The easiest way to setup robotoolbox is to store the token and url your .Renviron.

We are using the following environment variables KOBOTOOLBOX_URL and KOBOTOOLBOX_TOKEN to respectively store the server url and token. You can use the usethis R package and the usethis::edit_r_environ() function to add these environment variables to your .Renviron file.

KOBOTOOLBOX_URL="https://kobo.unhcr.org/"
KOBOTOOLBOX_TOKEN=xxxxxxxxxxxxxxxxxxxxxxxxxx

In an interactive session, you can use directly the kobo_setup function.

library("robotoolbox")
kobo_setup(url = "https://kobo.unhcr.org",
           token = "xxxxxxxxxxxxxxxxxxxxxxxxxx")

You can get your token manually through the KoBotoolbox web interface in your account settings. There is also the kobo_token function to do the same thing automagically from R.

token <- kobo_token(username = "cool_user_name",
                    password = "gRe@TP@$$WoRd",
                    overwrite = TRUE)

This token can then be used in the setup.

kobo_setup(url = "https://kobo.unhcr.org",
           token = token)

and it is possible to check the settings with the kobo_settings function.

kobo_settings()
## <robotoolbox settings>
##    KoBoToolbox URL: https://kobo.unhcr.org/
##    KoBoToolbox API Token: xxxxxxxxxxxxxxxxxxxxxxxxxx

Kobotoolbox projects

List your projects

All your Kobotoolbox projects also named assets can be checked and listed through the web interface.

Using robotoolbox, you can do something similar from R with the kobo_asset_list function. The result of a call to kobo_asset_list is a tibble and can be manipulated as such.

library("robotoolbox")
library("dplyr")
al <- kobo_asset_list()

al |>
  slice_head(n = 6)
uid name asset_type owner_username date_created date_modified submissions
a37LegD9joqVEDjaGaDrmP WCA Programme management capacity building and COMPASS improvement survey survey dickoa 2023-01-23 20:02:50 2023-01-27 12:34:52 1
a9bwEQcaWqWaA5hzkjRAWx UNHCR FORMULAIRE COLLECTE DES INCIDENTS DE PROTECTION - INTERSOS survey imrca 2021-03-05 08:04:06 2023-01-04 08:57:57 10876
aA9QwKdEXqfGiQCkPNzKBR RMS CAPI POUR TEST GISSE survey dickoa 2022-12-30 16:23:57 2022-12-30 17:22:02 1
aEZsFfwvADtcAGzuVhd6UL date_time test survey dickoa 2022-12-18 13:31:21 2022-12-18 13:31:42 1
aGHVFT8pFCikCsimq8NtRS contact data manh test survey dickoa 2022-12-18 12:54:02 2022-12-18 12:54:32 1
acoSWjKHdu8nUpFSuAdt2h 2021_UNHCR_CIV_Menage Rapatrie survey unhcr_ico 2021-11-10 10:33:14 2022-10-26 09:34:26 12984

KoboToolbox asset

You can also manipulate directly each asset (project) using kobo_asset. An asset is uniquely identified by a unique identifier: the uid.

uid <- "aEwTYNcU76UvLgiY89rPWm"
asset <- kobo_asset(uid)
asset
<robotoolbox asset>  aEwTYNcU76UvLgiY89rPWm 
  Asset name: Pan-African Capacity Building in R
  Asset type: survey
  Asset owner: rrc_rrrp19
  Created: 2022-01-06 14:33:35
  Last modified: 2022-01-28 12:31:28
  Submissions: 95

Since al, the list of assets, is a data.frame, we can subset it and select the uid of interest based on the other available metadata.

As an example, you can get the uid associated to the project with the largest number of submissions.

al |>
  slice_max(submissions) |>
  pull(uid)
[1] "a7XzRuPFn9j5WkT2mR6wbg"

And it’s then possible to get the associated asset object.

al |>
  slice_max(submissions) |>
  pull(uid) |>
  kobo_asset()
<robotoolbox asset>  a7XzRuPFn9j5WkT2mR6wbg 
  Asset name: form for stress test
  Asset type: survey
  Asset owner: tmneche
  Created: 2021-04-14 10:19:19
  Last modified: 2021-05-21 11:31:33
  Submissions: 118261

Kobotoolbox data

The main function on the robotoolbox package is kobo_data (or kobo_submissions its alias) and as the name suggests, it loads your data from the server.

We can illustrate its usage with data from a project named Multiple languages.

The first step, is to get uid of the project (or asset).

uid_ml <- al |>
  filter(name == "Multiple languages") |>
  pull(uid)
uid_ml
[1] "aYuTZn9vegi3Z49MXwKjep"

We can the use uid_ml that uniquely identify the Multiple languages asset to read its data

data_ml <- kobo_data(uid_ml)

data_ml |>
  select(start:uuid)
start end today full_name pet_yesno _id uuid
2022-01-07 10:39:47 2022-01-07 10:40:02 2022-01-07 Fatim 1 17735003 5388c680d19148828ea45913af820f30
2022-01-07 10:39:37 2022-01-07 10:39:47 2022-01-07 Jean-Pierre 0 17735002 5388c680d19148828ea45913af820f30
2022-01-07 10:38:29 2022-01-07 10:39:37 2022-01-07 إبراهيم 1 17735000 5388c680d19148828ea45913af820f30
2022-01-07 10:38:20 2022-01-07 10:38:29 2022-01-07 Michelle 1 17734995 5388c680d19148828ea45913af820f30
2022-01-07 10:37:39 2022-01-07 10:38:20 2022-01-07 Ahmad 0 17734994 5388c680d19148828ea45913af820f30

Kobotoolbox form

You can also check the form used using the kobo_form function

uid_ml |>
  kobo_form()
name list_name type label lang kuid choices
start NA start NA English (en) Cypi2O0NQ
end NA end NA English (en) F2Erecceg
today NA today NA English (en) 6WnNQUnlF
full_name NA text What is your name? English (en) pkEk0bzX7
full_name NA text Quel est votre nom ? Francais (fr) pkEk0bzX7
full_name NA text ما اسمك ؟ Arabic (ar) pkEk0bzX7
pet_yesno yesno select_one Do you have any pet? English (en) a1xXud0rG 1 , 1 , 1 , 0 , 0 , 0 , Yes , Oui , نعم , No , Non , لا , English (en) , Francais (fr), Arabic (ar) , English (en) , Francais (fr), Arabic (ar)
pet_yesno yesno select_one Avez-vous un animal de compagnie ? Francais (fr) a1xXud0rG 1 , 1 , 1 , 0 , 0 , 0 , Yes , Oui , نعم , No , Non , لا , English (en) , Francais (fr), Arabic (ar) , English (en) , Francais (fr), Arabic (ar)
pet_yesno yesno select_one هل تمتلك حيوانا أليفا ؟ Arabic (ar) a1xXud0rG 1 , 1 , 1 , 0 , 0 , 0 , Yes , Oui , نعم , No , Non , لا , English (en) , Francais (fr), Arabic (ar) , English (en) , Francais (fr), Arabic (ar)

Which is a representation in robotoolbox of the following xlsform

Survey questions

type name label::English (en) label::Francais (fr) label::Arabic (ar)
start start
end end
today today
text full_name What is your name? Quel est votre nom ? ما اسمك ؟
select_one yesno pet_yesno Do you have any pet? Avez-vous un animal de compagnie ? هل تمتلك حيوانا أليفا ؟

Choices

list_name name label::English (en) label::Francais (fr) label::Arabic (ar)
yesno 1 Yes Oui نعم
yesno 0 No Non لا