Graph databases are starting to be used to help not just the enterprise but also the Civil Service and Police force, says Neo Technology’s Emil Eifrem

Governments have many of the same data problems as commercial corporations do in the Internet Age, namely expansive and growing collections of data.

But public sector ICT leaders have other problems beyond data; constricted budgets – and the need to deliver systems that work at mass (regional and national, indeed international) scale. As in business, they need to connect disparate silos of information so as to create what traditional enterprise Business Intelligence (BI) has been striving for – a holistic, 360-degree view of the customer or in this case, the service user.

Relational database management systems (RDBMS) have been challenged by these demands. Our modern world couldn’t function without relational, nor without the continued contribution of even earlier database formats (flat-file/network, CICS et al), but nonetheless they are not well suited to manage these new large volumes of data.

And nor, for all of their incredible power, are Big Data Hadoop-style data engines necessarily. Big Data databases handle the scale better than RDBMS, but just like relational, they miss the last mile – namely relationships.

That’s important, as it’s relationships – the connections between people, places, events, etc., that are the hub of what happens in the real world. Public services depend on being able to spot these connections for legal or improved service delivery reasons. And I am suggesting, a new way of working with data, graph database technology, is the missing link here – the tool that can help the public sector with all these data issues.

Firstly, graph database technology helps when it comes to unifying multiple back-end data sources – of great interest in the public sector. With graph databases, civil servants love the fact that they can start to see patterns emerge by connecting multiple legal, welfare and demographic datasets thanks to the connections and links graph technology can create for them.

Data-based policy

Let’s consider a real example: how graph database technology is enabling functionality that both RDBMS and Big Data/Hadoop technology would struggle to produce. A G8 country’s primary immigration agency needed help to visualise relationships and connections. In the first instance, this was around case management, helping its civil servants work with individual cases of potential interest to border control. Immediate access to such information is the potential difference between stopping a criminal at the border or letting them through there and then, while the relational database is still spinning away, crunching data too complex for it to process in real-time.

The government customer is deriving insights which aren’t just helping it deal with immediate issues, but they are becoming the data-driven basis for truly informed policy creation. Why suggest initiatives if they’re not based on actual user behaviour, after all? As a result, the team is talking to other departments within this country’s state administration about the results they’re seeing – which means the database technology could soon be enabling a new, highly responsive informal learning system in more than one national application.

That learning system is also starting to incorporate one of the most important data sources for all of us – social media. Our customer isn’t storing the billion Tweets or Instagram posts released every day. Instead, it’s doing that collation the smart way, working with top-level metadata to spot hidden patterns and networks of interest. Graph databases use fuzzy logic which is really helpful here, as it’s so effective at spotting how slightly variant name spellings all refer back to the same person, something classical business databases would find challenging to produce the same results.

Improved public services could well result

Judging by these experiences and the kinds of conversations we have with public sector ICT and policy people all over the globe, graph databases could really make a contribution in the drive to deliver cheaper, but more integrated, digital public services. Real, achievable benefits for the public from this kind of work will be more secure border control, safer living environments, protection from fraudulent behaviour.

Judged in that light, e-gov graph database exploitation could well be one of the tools budget-conscious civil servants should be using.

The author is co-founder and CEO of Neo Technology, the company behind Neo4j (http://neo4j.com/), the world’s leading graph database