ChEMBL Resources

Resources:
ChEMBL
|
SureChEMBL
|
ChEMBL-NTD
|
ChEMBL-Malaria
|
The SARfaris: GPCR, Kinase, ADME
|
UniChem
|
DrugEBIlity
|
ECBD

Wednesday, 21 January 2015

The ChEMBL Roadshow: Part II



After the very successful US East Coast ChEMBL Roadshow, we (Anne and George) will be on the road once again next week to spread the word on ChEMBL, myChEMBL and SureChEMBL. This time we will visit the US West Coast and specifically these venues:

  • Tuesday 27th Jan: University of New Mexico, Albuquerque.
  • Wednesday 28th: MolSoft/UCSD/Scripps, San Diego. See also here.
  • Thursday 29th: IBM Research Centre, San Jose.
  • Friday 30th: UCSF, San Francisco.


If you are nearby and would like to attend or meet us for a chat, please get in touch. 

We are grateful to SMSdrug.net for funding. 


George & Anne


ChEMBL 20 schema

For those who just can't wait to update their code... Here is a picture of the new ChEMBL_20 schema.


Monday, 5 January 2015

ChEMBL 20 coming soon...


Happy New Year for 2015 from the ChEMBL group!

Release 20 of the ChEMBL database will be happening around the end of the month, and for those who can't wait, here's a preview of the exciting new features you can expect to find there:

  • HELM notation - we have developed an implementation of the Pistoia Alliance's HELM standard for biotherapeutics and will be supplying HELM notation for just under 20K peptides (previously represented by mol files). We will also make our monomer library available in case others wish to use it to generate their own HELM notation.
  • Structural alerts - in place of the old 'Med Chem Friendly' flag used in ChEMBL, we now have an extensive set of structural alerts calculated for the ChEMBL compounds. The data set includes eight different sets of alerts (including sets published by Pfizer, Glaxo, BMS, University of Dundee, NIH MLSMR and PAINS filters) providing more than 1100 distinct SMARTS. Alerts found for a given compound can be viewed on the interface and the full data set will be included in the database downloads.
  • Cell Report Cards - we will now provide ChEMBL IDs and Report Card pages for cells and cell-lines, as well as a keyword search facility. This will enable users to quickly identify all assays that have been performed in a particular cell-line and will also provide cross references to other resources such as the LINCS project.
  • HRAC/FRAC/IRAC classification - for known herbicides, fungicides and insecticides, we provide their classification according to HRAC/FRAC/IRAC. This gives an indication of the mechanism of action of each compound. The classification can be found on the compound report card for these crop protection chemicals.
  • New bioactivity data - in addition to the usual updates of bioactivity data from scientific literature and PubChem BioAssay we have 12 new sets of screening results from the MMV Malaria Box and an extensive set of in vitro DMPK and physiochemical property data for more than 5,700 publicly disclosed drugs and compounds deposited by AstraZeneca.
  • There will be some schema changes to incorporate these new features, but these will be additions and are unlikely to break existing code.
Watch this space!


The ChEMBL Team

Monday, 1 December 2014

Accessing web services with cURL


ChEMBL web services are really friendly. We provide live online documentation, support for CORS and JSONP techniques to support web developers in creating their own web widgets. For Python developers, we provide dedicated client library as well as examples using the client and well known requests library in a form of ipython notebook. There are also examples for Java and Perl, you can find it here.

But this is nothing for real UNIX/Linux hackers. Real hackers use cURL. And there is a good reason to do so. cURL comes preinstalled on many Linux distributions as well as OSX. It follows Unix philosophy and can be joined with other tools using pipes. Finally, it can be used inside bash scripts which is very useful for automating tasks.

Unfortunately first experiences with cURL can be frustrating. For example, after studying cURL manual pages, one may think that following will return set of compounds in json format:


But the result is quite dissapointing...


The reason is that --data-urlencode (-d) tells our server (by setting Content-Type header) that this request parameters are encoded in "application/x-www-form-urlencoded" - the default Internet media type. In this format, each key-value pair is separated by an '&' character, and each key is separated from its value by an '=' character for example:


This is not the format we used. We provided our data in JSON format, so how do we tell the ChEMBL servers the format we are using? It turns out it is quite simple, we just need to specify a Content-Type header:


If we would like to omit the header, correct invocation would be:


OK, so request parameters can be encoded as key-value pairs (default) or JSON (header required). What about result format? Currently, ChEMBL web services support JSON and XML output formats. How do we choose the format we would like the results to be returned as? This can be done in three ways:

1. Default - if you don't do anything to indicate desired output format, XML will be assumed. So this:


will produce XML.

2. Format extension - you can append format extension (.xml or .json) to explicitly state your desired format:


will produce JSON.

3. `Accept` Header - this header specifies Content-Types that are acceptable for the response, so:


will produce JSON.

Enough boring stuff - Lets write a script!


Scripts can help us to automate repetitive tasks we have to perform. One example of such a task would be retrieving a batch of first 100 compounds (CHEMBL1 to CHEMBL100). This is very easy to code with bash using curl (Note the usage of the -s parameter, which prevents curl from printing out network and progress information):


Executing this script will return information about first 100 compounds in JSON format. But if you carefully inspect the returned output you will find that some compound identifiers don't exist in ChEMBL:


We need to add some error handling, for example checking if HTTP status code returned by server is equal to 200 (OK). Curl comes with --fail (-f) option, which tells it to return non-zero exit code if response is not valid. With this knowledge we can modify our script to add error handling:


OK, but the output still looks like a chaotic soup of strings and brackets, and is not very readable...

Usually we would use a classic trick to pretty print json - piping it through python:


But it won't work in our case:



Why? The reason is that python trick can pretty-print a single JSON document. And what we get as the output is a collection of JSON documents, each of which describes different compound and is written in separate line. Such a format is called Line Delimited JSON and is very useful and well known.

Anyway, we are data scientists after all so we know a plenty of other tools that can help. In this case the most useful is jq - "lightweight and flexible command-line JSON processor", kind of sed for JSON.

With jq it's very easy to pretty print our script output:



Great, so we finally can really see what we have returned from a server. Let's try to extract some data from our JSON collection, let it be chemblId and molecular weight:



Perfect, can we have both properties printed in one line and separated by tab? Yes, we can!



So now we can get the ranking of first 100 compounds sorted by their weight:




Exercises for readers:

1. Can you modify compounds.sh script to accept a range (first argument is start, second argument is length) of compounds?
2. Can you modify the script to read compound identifiers from a file?
3. Can you add a 'structure' parameter, which accepts a SMILES string. When this 'structure' parameter is present, the script will return similar compounds (you can decide on the similarity cut off or add an extra parameter)?



Monday, 10 November 2014

Finding key compounds in med. chemistry patents: The open way


A couple of us attended the 3rd RDKit UGM, hosted by Merck in Darmstadt this year. It was an excellent opportunity to catch up with RDKit developments and applications and meet up with other loyal "RDKitters".

I presented a talk-torial there and went through an IPython Notebook, which some of you may find useful. It uses patent chemistry data extracted from SureChEMBL and after a series of filtering steps, it follows a few "traditional" chemoinformatics approaches with a set of claimed compounds. My ultimate aim was to identify "key compounds" in patents using compound information alone, inspired by papers such as this and this. The crucial difference is that these authors used commercial data and software, where in this implementation everything is free and open. At the same time, I wanted to show off what the combination of pandas, scikit-learn, mpld3, Beaker, RDKit, IPython Notebook and SureChEMBL can do nowadays (hint: a lot). 

So, here is the Notebook and here are the associated slides which give a bit of background and context. 

Obviously, the logic and steps can be reimplemented with other toolkits or workflow tools, such as KNIME


George




Wednesday, 5 November 2014

Using ChEMBL web services via proxy.




It is common practice for organizations and companies to make use of proxy servers to connect to services outside their network. This can cause problems for users of the ChEMBL web services who sit behind a proxy server. So to help those users who have asked, we provide the following quick guide, which demonstrates how to access ChEMBL web services via a proxy.

Most software libraries respect proxy settings from environmental variables. You can set the proxy variable once, normally HTTP_PROXY and then use that variable to set other related proxy environment variables:


Or if you have different proxies responsible for different protocols:


On Windows, this would be:



If you are accessing the ChEMBL web services programmatically and you prefer not to clutter your environment, you can consider adding the proxy settings to your scripts. Here are some python based recipes:


1. Official ChEMBL client library


If you are working in a python based environment, we recommend you to use our client library (chembl_webresource_client), for accessing ChEMBL web services. It already offers many advantages over accessing the ChEMBL web services directly and handling proxies is yet another. All you need to do is configure proxies once and you are done:



2. Python requests library


If you decide to use requests, you have to add 'proxies' parameter to every 'get' and 'post' function call:



3. Python urllib2 library


Finally, in the lowest level library, 'urllib2' you can set a ProxyHandler and register it to URL opener:



We would like to thank Dr. Christine Rudolph for the idea and providing code snippets.

Tuesday, 4 November 2014

An overview and invitation to contribute to ChEMBL curation with PPDMs

PPDMs has been in the making for more than a year and is a follow-up on a conference paper we published in 2012. As in 2012, our objective is to map small molecule binding sites to protein domains, the structural units that form recurring building blocks in the evolution of proteins. An application note describing PPDMs is just out in Bioinformatics.

Mapping small molecule binding to protein domains

The mapping facilitates the functional interpretation of small molecule-protein interactions - if you understand which domain in a protein is targeted, you are in a better position to anticipate the downstream effect.  Mapping small molecule binding to protein domains also provides a technical advantage to machine-learning approaches that incorporate protein sequence information as a descriptor to predict small molecule bioactivity. Reducing the sequence descriptor to the part that mediates small molecule binding increases the informative content of the descriptor. This is best exemplified by the domain-poisoning problem, illustrated below.
Result of a hypothetical query using as input the rat Tyrosine-protein phosphatase Syp (P35235) - and one of the hits, retrieved from a BLAST query against the ChEMBL target dictionary - the rat Tyrosine-protein kinase SYK (Q64725). The significant e-value for this query results from high scoring alignments of the SH2 domains. At the same time, the overlap between small molecules binding both proteins is expected to be low.

A simple heuristic

For individual experiments, it is often quite trivial to decide which domain was targeted. For example, medicinal chemists know whether their compound is a kinase inhibitor or one of a handful of SH2 inhibitors. This knowledge, while easily gleaned by the expert, is implicit and cannot be accessed programmatically. Hence we were motivated to implement a solution that could achieve this across as many measured bioactivities as possible.

Our initial implementation of mapping small molecules to protein domains consisted of a simple heuristic: Identify domains with known small molecule interaction and use these domains as a look-up when mapping measured bioactivities to protein domains. This process is illustrated in the figure below.

A catalogue of validated domains was extracted from assays against single-domain proteins (step 1, 2) and projected onto measured bioactivities in ChEMBL (step 3). Three possible outcomes are: i) A successful mapping if exactly one of the Pfam-A domain models from the catalogue matches the sequence; ii) No mapping if none of the Pfam-A domain models from the catalogue match the sequence; iii) A conflicting mapping if multiple domain models from the catalogue match the sequence.
Despite its simplicity, this method works surprisingly well, owing to the fact that protein domains that are relevant to drug discovery are prioritised in Pfam-A model curation. Another factor that contributes here is the conservative route taken by many drug discovery projects that focus on targets that are in well characterised protein families. However, as illustrated by the cases labelled ii) and iii), some constellations are not covered by the simple heuristic.

A public platform to review and improve mappings


Measured activities in ChEMBL falling into category iii) from the illustration above amount to only a fraction of the total but often reflect interesting biology. DHFR-TS for example is a multi-functional enzyme combining both a DHFR and Thymidylate_synt domain that occurs in the group of bikonts, which includes Trypanosoma and Plasmodium. In humans (and all metazoa), these domains occur as separate enzymes.
Small molecule inhibitors exist for both domains, DHFR (yellow, with Pyrimethamine) and Thymidylate synthase (blue, with Deoxyuridine monophosphate).
We built PPDMs as a platform to resolve such cases. PPDMs aggregates information that supports manual mapping assignments based on medicinal chemistry knowledge. New mappings can be  committed to the PPDMs logs and then transferred to the ChEMBL database in future releases.

The Conflicts section on the website summarises conflicts (cases that correspond to category iii as discussed above) that were encountered when the mapping was applied to measured activities in the ChEMBL database and offers an interface to resolve them.

The Evidence section provides the full catalogue of domains for which we found evidence of small molecule binding. Evidence for the majority of domains in this list is provided in the form of measured bioactivities in ChEMBL, while in a few cases we provide a reference to the literature. These are cases where well-known domains occur exclusively in multi-domain architectures, such as 7tm_2 and 7tm_3. The catalogue can be downloaded in full from this section.

PPDMs also provides logs of individual assignments - these can be queried by date, user and comments left when the assignment was made. A log of all assigned mappings can be downloaded from this section. Another way to review assigned mappings is through the Resolved section, where assignments are grouped by domain architecture.

We invite everyone with an interest in the matter to sign up with PPDMs, whether it's simply for playing around, resolving remaining conflicts, or reviewing existing assignments.  Please get in touch and we'll sort out a login for you!

felix

Tuesday, 28 October 2014

Paper: PPDMs – A resource for mapping small molecule bioactivities from ChEMBL to Pfam-A protein domains


We've just published a Open Access paper in Bioinformatics on an approach to annotate the region of ligand binding within a target protein. This has a lot of applications in the use of ChEMBL, in particular providing greater accuracy in mapping functional effects, improving ligand-based target prediction approaches, and reducing false positives in sequence/target searching of ChEMBL. Where next for this work - well annotating to a site-specific level would be a good thing to implement (think about HIV-1 RT with the distinct nucleoside and non-nucleoside sites).

Here's the abstract...

Summary: PPDMs is a resource that maps small molecule bioactivities to protein domains from the Pfam-A collection of protein families. Small molecule bioactivities mapped to protein domains add important precision to approaches that use protein sequence searches alignments to assist applications in computational drug discovery and systems and chemical biology. We have previously proposed a mapping heuristic for a subset of bioactivities stored in ChEMBL with the Pfam-A domain most likely to mediate small molecule binding. We have since refined this mapping using a manual procedure. Here, we present a resource that provides up-to-date mappings and the possibility to review assigned mappings as well as to participate in their assignment and curation. We also describe how mappings provided through the PPDMs resource are made accessible through the main schema of the ChEMBL database.

Availability: The PPDMs resource and curation interface is available at https://www.ebi.ac.uk/chembl/research/ppdms/pfam_maps

The source-code for PPDMs is available under the Apache license at https://github.com/chembl/pfam_maps

Source code is available at https://github.com/chembl/pfam_map_loader to demonstrate the integration process with the main schema of ChEMBL.

Monday, 27 October 2014

Django model describing ChEMBL database.





TL;DR: We have just open sourced our Django ORM Model, which describes the ChEMBL relational database schema. This means you no longer need to write another line of SQL code to interact with ChEMBL database. We think it is pretty cool and we are using it in the ChEMBL group to make our lives easier. Read on to find out more....



It is never a good idea to use SQL code directly in python. Let's see some basic examples explaining why:


Can you see what is wrong with the code above? SQL keyword `JOIN` was misspelled as 'JION'. But it's hard to find it quickly because most of code highlighters will apply Python syntax rules and ignore contents of strings. In our case the string is very important as it contains SQL statement.

The problem above can be easily solved using some simple Python SQL wrapper, such as edendb. This wrapper will provide set of functions to perform database operations for example 'select', 'insert', 'delete':


Now it's harder to make a typo in any of SQL keywords because they are exposed to python so IDE should warn you about mistake.

OK, time for something harder, can you find what's wrong here, assuming that this query is executed against chembl_19 schema:


Well, there are two errors: first of all `molecule_synonyms` table does not have a `synonims` column. The proper name is `synonyms`. Secondly, there is table name typo  `molecule_synonyms`.

This kind of error is even harder to find because we are dealing with python and SQL code that is syntactically correct. The problem is semantic and in order to find it we need to have a good understanding of the underlying data model, in this case the chembl_19 schema. But the ChEMBL database schema is fairly complicated (341 columns spread over 52 tables), are we really supposed to know it all by heart? Let's leave this rhetorical question and proceed to third example: how to query for compounds containing the substructure represented by 'O=C(Oc1ccccc1C(=O)O)C' SMILES:

For Oracle this would be:


And for Postgres:


As you can see both queries are different, reasons for these differences are:
  1. Differences in Oracle and Postgres dialects
  2. Different chemical cartridges (Accelrys Direct and RDKit)
  3. Different names of auxiliary tables containing binary molecule objects
These queries are also more complicated than the previous examples as they require more table joins and they make calls to the chemical cartridge-specific functions.

The example substructure search queries described above are similar to those used by the ChEMBL web services, which are available on EBI servers (Oracle backend) and in the myChEMBL VM (PostgreSQL backend). Still, the web services work without any change to their code. How?

All of the problems highlighted in this blogpost can be solved by the use of a technique known as Object Relational Mapping (ORM). ORM converts every table from database (for example 'molecule_dictionary') into Python class (MoleculeDictionary). Now it's easy to create a list of all available classes in Python module (by using 'dir' function) and check all available fields in class which corresponds to columns from SQL tables. This makes database programming easier and less error prone. The ORM also allows the code to work in a database agnostic manner and explains how we use the same codebase with Oracle and PostgreSQL backends.

If this blogpost has convinced you to give the ORM approach a try, please take a look at our ChEMBL example also included in myChEMBL:

Wednesday, 22 October 2014

myChEMBL 19 Released



                     
We are very pleased to announce that the latest myChEMBL release, based on the ChEMBL 19 database,  is now available to download. In addition to the extra data, you will also find a number a great new features. So what's new then?

More core chemoinformatics tools

We have included OSRA (Optical Structure Recognition), which is useful for extracting compound structures from images. OSRA can be accessed from the command line or by very convenient web interface, provided by Beaker (described below). We've also added OpenBabel - another great open source cheminformatics toolkit. This means you can now experiment with both RDKit and OpenBabel and use whichever you prefer.

ChEMBL Beaker

myChEMBL now ships with a local instance the ChEMBL Beaker service. For those not familiar with Beaker, the service provides users with an array of chemoinformatics utilities via a RESTful API. Under the hood, Beaker is using RDKit and OSRA to carry out its methods. With the addition of Beaker in myChEMBL, users can now carry out the following tasks in secure local environment:
  • Convert chemical structure bewteen multiple formats
  • Extract compound information from images and pdfs
  • Generate compound images in raster (png) and vector (svg) forms
  • Generate HTML5 ready representation of compound structure
  • Generate compound fingerprints
  • Generate compound descriptors
  • Identify Maximum Common Substructure
  • Compound standardisation
  • Lots of more calculations

 

New IPython notebooks

We have written a number of new IPyhthon notebooks, which focus on a range of cheminformatics and bioinformatic topics. The topics covered by the new notebooks include:
  • Introduction on how to use ChEMBL Beaker
  • Using the Django ORM to query the ChEMBL database
  • Introduction to BLAST and creation of a simple Druggability Score
  • Introduction to machine learning
  • Analysis of SureChEMBL data, focused on identifying the MCS core identified in a patent 
  • Extraction and analysis of ChEMBL ADME data 

We have also updated the underlying Ubuntu VM to 14.04 LTS, which also required us to make a number of changes the myChEMBL installation. To see how these changes and new additions have effected a bare metal installation of myChEMBL, head over the myChEMBL github repository.

 

Installation

There are 2 different ways we recommend for installing myChEMBL:
  1. Follow the instructions in the INSTALL file on the ftpsite. This will import the myChEMBL VM into VirtualBox
  2. Use Vagrant to install myChEMBL. See this earlier blogpost for more details, but the command to run is:
vagrant init chembl/myChEMBL && vagrant up

   If you already have myChEMBL_18 installed via Vagrant, instead of running 'vagrant box update', we strongly recommend running: 

vagrant box remove chembl/myChEMBL
vagrant init chembl/myChEMBL && vagrant up

Future plans

The myChEMBL resource is an evolving system and we are always looking to add new open source projects, tools and notebooks. We would be really interested to hear from users about what they would like to see in future myChEMBL releases, so please get in touch if you have any suggestions. (Just so you know, we already have a couple of ideas for myChEMBL 20).

We hope you find this myChEMBL update useful and if you spot any issues or have any questions let us know.

The myChEMBL Team