Online job postings data pilot: Q&A for developers

LMI for All is an online data service, delivered via an API (application programming interface), that provides access to high quality labour market data to third party developers. We want developers to use the data to power applications and websites that help people to make good careers decisions.

LMI for All is providing access to real-time labour market data on a pilot basis from the beginning of January to the end of March 2016. Access is free of charge, subject to terms and conditions (see below). You can get in touch now to register your interest in the data and find out more.

The following document provides answers to key questions relating to the pilot. If you have a query that is not covered below please get in touch via

button blue white text

To refresh your memory about LMI for All or to find out more about the project, please visit http://www.lmiforall.org.uk/

What data will you be making available?

We are providing access to counts of UK online job postings, classified by occupation (standard occupational classification), industry and location (UK nation, English region, local travel-to-work area).

The counts relate to job postings made in the recent past (e.g. previous 90 days) and the data will be updated on a rolling basis each day to ensure currency.

Why are you providing access to this dataset?

Real-time job postings have enormous potential in the field of labour market intelligence, as an ever-increasing proportion of job openings are posted on the internet. We want to test the benefits of widening access to this kind of analysis, particularly in the context of careers decision making.

We believe that we can achieve a significant “public good” by making the data widely and freely available, particularly since there are currently no official statistics available on the occupational profile of vacancies.

We will use the learning from our pilot to determine whether to roll-out real-time data via LMI for All on an ongoing basis. We will also document and publish our learning for wider consumption.

Why is this dataset useful?

When making a choice about the right career path for them, it’s helpful for an individual to get an impression of the number of opportunities that are currently available for a job or jobs that they are interested in. Is there demand for people who are qualified to do the job? Quite often the individual is focused on a particular geographic area. They may want to know whether they can pursue a particular career in their local area and whether there are sufficient opportunities for them to do so.

Where do the data come from?

The data are being sourced from Burning Glass, a leading supplier of real-time labour market intelligence. Burning Glass are a US-based company who are establishing a strong presence in the UK.

We are interested in other job postings data such as skills required for particular jobs. Why is this not being made available?

The range of data that we can make available through the trial is limited by budget constraints. The more variables we include the more expensive it becomes. If you have an opinion about the data that is most valuable for careers purposes we would like to hear from you.

How can I be sure of the quality of the data?

Burning Glass’ data are compiled from thousands of sources of job postings in the UK, including online job boards and company websites. The data are collected and parsed locally i.e. within the UK and sites are vetted in advance to ensure they are a valid source of job opportunities.

New job postings are identified, processed, and added to the Burning Glass database each day.

Within job postings more than 70 data elements are parsed, extracted and coded including job title which is used to map to an occupation and employer which is used to assign an industry code. Burning Glass use a curated set of tens of thousands of business rules to appropriately assign a job into an occupation.

Robust mechanisms have been put in place to deduplicate job advertisements to ensure that the values presented are based on unique opportunities rather than an aggregation of all posting activity undertaken by recruiting and staffing agencies and other firms.

Are there any restrictions on how we can use the data?

There are additional restrictions placed on the use of this dataset, beyond what is already contained in our standard terms and conditions for LMI for All.

In line with the overall aim of LMI for All, developers (including commercial organisations) may use the data as an input to applications and websites that are designed to support individuals in making choices about careers and learning options. The data may not be used for other commercial purposes e.g. you may not download the data in bulk for resale.

Each organisation will be vetted to ensure that its proposed use of the data is in keeping with the spirit of this objective.

Full terms and conditions will be provided when you apply for access to the data.

How long does the pilot run for and what happens afterwards?

Our current plan is to run a short pilot to from the beginning of January 2016 to 31 March 2016 to gauge demand from third party developers.

Depending on the level of demand and interest we may seek to offer data of this kind on a longer term basis or to extend the duration of the pilot.

In any event, we will not cut off access to the data on the 31 March; it will continue to be available afterwards but it will not be updated on a daily basis.

What do I need to do next?

If you are interested in exploring the dataset and potentially incorporating it into an application or website please email 

button blue white text

and we will issue you with a unique passcode.

Who can I contact with technical queries about the API in general or specifically about the real-time data?

Please contact 

button blue white text

in the first instance.

A video tutorial: Getting started with the LMI for All API

We are recording a series of video tutorials to support the development of applications based on the LMI for All API. This first, introductory video, shows how to use JQUERY to search for Standard Occupational Classification (SOC) codes for different jobs and applications and how to display the results. The code is available in a GitHub repository.

If you have any questions please use the comment box below or the forum on this site.

Settle for a job or aim for a career? How LMI for All can help you

Paul McKelvie is a UKCES Commissioner and works on the LMI for All project. In this blog post he explains how LMI for All can be used to help people develop a career.

Is a ‘career’ a string of jobs which we retrospectively say were all part of a masterplan? I don’t think so, certainly not for most of us. For me, the key element which makes a string of jobs a career is the affirmative decision at the outset: the chosen occupation, the pathway to it, and the overriding motivation to pursue it. These are unique to each and every one of us.

Yet when we weigh up a career, we visualise in terms of concrete information as well as values and purpose. We ask: how much will I be paid? How many hours will I work? How many opportunities will there be? How do I get into this occupation?  This is the information that helps us make well informed career decisions.

LMI for All exists to help with that part of the decision. It is an open access data service, which brings together robust national sources of labour market information (LMI) and makes these freely available to web developers. The aim is to inform careers decision makers by providing accurate and reliable intelligence about jobs.

LMI for All compiles and quantifies data from multiple sources – the Labour Force Survey, the Annual Survey of Hours and Earnings, and UKCES’ own Working Futures to name but three. LMI for All puts these varied datasets into one place and indexes them by Standard Occupational Code (SOC), a system for classifying job titles.

This is then made freely available, with no charge and only the most necessary terms and conditions.

Why is it so important to allow free and uninhibited access? Because allowing data to be accessed and re-used freely ensures that creative developers can use it to power apps or websites that present labour market data in innovative, striking ways. Open data means that everyone, from an established tech company to an aspiring young web designer can experiment with LMI, and discover new, better ways of presenting it.

The virtue of free, open access is evident in the variety of ways our users have presented LMI for All data. Plotr, icould, and SACU are among the users of LMI for All and all have found different ways to showcase the data.

Tomasz Florczak, the winner of our CareerHack competition, created a ‘Career Advisor’ app which uses data from LMI for All to compare careers based on geographical area. This, incidentally, has just been released onto the market.

Schools and colleges can directly access information via our LMI for All ‘Careerometer’ widget. It’s free, customisable and easy to insert into your webpage with just a few lines of computer code.

With a focus on robust, reliable data and regular new features, campaigns and users, LMI for All will continue to supply young decision makers with the information they need to turn that string of jobs into a fulfilling, successful career. Can I encourage you to make best use of it

LMI for All Report

March, 2015, marked the end of the second phase of development for LMi for All. The report of the phase two work summarised the outcomes as follows.

Overall, the first three years of pilot development of LMI for All has been successful in achieving three key goals:

  • The development of a comprehensive data offer;
  • the implementation of robust, secure, fit-for-purpose technical infrastructure;

and

  • An increased awareness and understanding throughout the stakeholder community of its existence as a high quality, free resource.

Whilst the database has been developed to a level where it can be, and is being, harnessed by a range of stakeholder groups for various purposes, further areas for development include: updating current databases and adding additional databases relevant to supporting decisions about learning and work; further enhancement and testing of the technical infrastructure; and additional work with stakeholder groups to ensure the potential, together with the likely processes of engagement, are understood and can be implemented within organisational contexts.

You can download the full report here: LMI_for_All_Final_report.