neuPrint includes both an API which provides a range of queries as well as the option to send custom queries written in the Cypher query language of the Neo4j graph database.
It is probably worth making queries via the API if they will solve your problem. However custom queries offer maximum flexibility.
It is a great idea to wrap your queries in a function. This will make your code cleaner and more reusable. But do check the documentation to ensure that there isn’t already a neuprintr function that does the job. If there is something that almost looks correct, then consider asking us to adapt it!
We’ll start by giving a basic example query function. Most of the time you should just be able to edit this example without worrying about the details. The subsequent sections give you some information about those details.
This function takes one or more bodyids as input and returns the name of the neurons.
neuprint_get_neuron_names <- function(bodyids, dataset = NULL, all_segments = FALSE,
conn = NULL, ...) {
bodyids = neuprint_ids(bodyids, conn = conn, dataset = dataset)
all_segments.json = ifelse(all_segments,"Segment", "Neuron")
cypher = sprintf("WITH %s AS bodyIds UNWIND bodyIds AS bodyId MATCH (n:`%s`) WHERE n.bodyId=bodyId RETURN n.instance AS name",
id2json(bodyids),
all_segments.json)
nc = neuprint_fetch_custom(cypher=cypher, conn = conn, dataset = dataset, ...)
d = unlist(lapply(nc$data,nullToNA))
names(d) = bodyids
d
}bodyids is the main input argument. These typically come in as either character vector or numeric format. By passing the argument to the neuprint_ids() function you can also enable a range of queries to define the bodyids. The internal function id2json() is eventually used to look after formatting them appropriately for the Neo4j cypher query.
In this function, cypher is the actual query written in the Cypher query language. One helpful tip. You can press the i key in neuPrint explorer to reveal the cypher query!
Many functions that operate on neurons will have an argument controlling whether they operate only for larger objects (aka Neuron) or also on fragments (aka Segment). Restricting queries to Neuron can result in big speed-ups in some cases. You can provide an all_segments argument to handle this option.
Finally the results that come back from neuprint_fetch_custom are typically in a big list object. For simple results, you can just unlist() this to make a vector. In this case a function nullToNA is first applied in order to ensure that any values that come back as NULL are converted to NA; this is necessary to ensure that you can make a vector - vector objects can only contain NAs not NULLs.
As you can see this step uses the sprintf function to interpolate variables into a string. You could do this in other ways (e.g. the paste() function or the glue() package). You may need to watch out for quoting issues if you should need to use single or double quotes inside your queries.
You need to be a little careful with the handling of body ids. The basic advice is to pass them to neuprint_ids() on entry to your function. This will ensure that there is at least one id (unless mustWork=FALSE) and convert to character vector. It also allows simple queries (by default exact matches against the type field). Finally it will by default ensure that only unique ids are passed in. This is nearly always what you want, but be careful.
In more detail, body ids are 64 bit integers (often called bigints), which are commonly used as keys in databases. However like many programming languages R does not have a native 64 bit integer type. R’s default numeric format is double width floating point. This can exactly handle numbers with up to 53 bits of precision.
However, the biggest integer (here data is represented in signed format int64) that we need to worry about is (2^63)-1 = 9,223,372,036,854,775,807. This cannot be represented as a numeric. I have yet to spot a body id in this upper range, but the neuprint_ids function will take care that you do not use one. If you need to specify a large bodyid as input to your function then you should insist on
bit64::integer64()
The latter seems better in theory (since it is more compact) than using character vectors, but it relies on an add-on package (bit64) rather than base R and I have seen subtle errors when integer64 objects lose their class. In particular you cannot turn them into a list or run lapply without losing their class. Therefore I recommend using character vectors where possible.
Bottom line
neuprint_ids() to your incoming body ids to enable a range of simple queries and check that your input looks sensible.id2json() on your body ids when building your cypherneuprint_ids(), then at least use id2char to put them into the most robust standard form (a character vector).In order to make a query, you need to specify which neuPrint server you want to talk to as well as the dataset you would like to use. The server is specified by a neuprint_connection() object. 99% of the time you will not need to do anything as this will be handled transparently by a one time user setting to specify their preferred server and authentication token.
The same is true of the dataset parameter, with the additional feature that that neuprint_fetch_custom() will internally check for a default dataset for the current server (as defined by the conn connection object).
However your function must allow people to specify both of these things if they wish. Therefore your function should look something like this in outline:
myquery <- function(query, conn=NULL, dataset=NULL, ...) {
neuprint_fetch_custom(query, conn=conn, dataset=dataset, ...)
}Internally neuprint_fetch_custom(), which will look after the dataset argument, and then go on to call the low level neuprint_fetch(), which will ensure that the connection object is valid (or use neuprint_login() to make one).
The sample function just ran unlist() on the list returned by neuprint_fetch_custom(). This is fine in many cases. You can also pass the simplifyVector argument to neuprint_fetch_custom() and this will clean up many forms of list. See jsonlite::fromJSON() for details of how this operates.
If you are getting a nested list specifying a table (i.e. a data.frame or spreadsheet like result) then there is a function neuprint_list2df() which will do a lot of the heavy lifting. It is strongly recommended to make use of this whenever possible. See neuprint_get_meta() for an example. Note that neuprint_list2df() has an argument with default stringsAsFactors=FALSE to ensure that columns in the output data.frame will be character vectors unless you take steps to make them factors.
Feel free to ask for help on the nat-user google group or by making an issue.