PHP Packages Graph

PHP Packages Graph started as a research project to understand the PHP ecosystem beyond package names, download counts, and individual repository metadata.

The goal was not simply to collect data from Packagist. The real goal was to transform Composer package metadata into a graph model where dependency relationships, package influence, licensing patterns, stability signals, and maintenance trends could be explored as part of one connected ecosystem.

PHP Packages Graph is designed as a data-to-analysis pipeline and combines backend scripting, data engineering, graph databases, dependency analysis, software ecosystem research, and exploratory data analysis.

The Problem

The core problem behind PHP Packages Graph was not access to PHP packages. The real problem was understanding how the PHP open-source ecosystem is structured.

Packagist and Composer make it easy to install packages, but they do not directly answer deeper ecosystem questions:

Which packages are the most structurally important?

Which libraries are dependency bottlenecks?

Which packages are mostly used in development?

Which licenses dominate the ecosystem?

Which old packages are still maintained?

Which packages have stable releases?

Which package types are common or unusual?

A package manager is excellent at resolving dependencies for one project. It is not designed to explain the dependency structure of an entire ecosystem.

That matters because modern PHP applications often depend on dozens or hundreds of direct and transitive packages. Some libraries become invisible infrastructure. They may not be product-facing, but they sit under a large part of the ecosystem.

In practice, understanding the PHP ecosystem becomes a graph problem.

A package can require another package. A vendor can own many packages. A package can conflict with, replace, provide, suggest, or depend on another package only in development. These relationships are not secondary metadata. They are the structure of the ecosystem itself.

That turned PHP Packages Graph from a simple Packagist data collection script into a graph-based research system.

Research Direction

PHP Packages Graph started with a simple idea:

Build a dependency graph of PHP packages published on Packagist.

That idea quickly became more interesting. A useful system had to do more than fetch package JSON files. It needed to:

• collect package names from Packagist;

• fetch detailed package metadata;

• store the raw dataset locally for replay and inspection;

• normalize package records across versions;

• extract dependencies from require, require-dev, conflict, provide, replace, and suggest;

• model vendors, packages, and dependency relationships in Neo4j;

• preserve useful metadata such as licenses, authors, downloads, repository links, package type, abandoned status, and update dates;

• expose the graph through Cypher queries;

• support research questions around influence, stability, licensing, and maintenance.

The engineering challenge was to design a reproducible pipeline capable of moving from Packagist’s package metadata to a graph database that can be queried like an ecosystem map.

System Architecture

PHP Packages Graph is organized around four main layers:

Each layer has a clear responsibility.

The collection layer solves data acquisition. The modeling layer solves validation and normalization. The graph store solves persistence and relationship modeling. The analysis layer turns the graph into research questions.

The system follows this flow:

Collection   -> fetches package names and package metadata

Modeling     -> validates and aggregates package information

Graph Store  -> stores vendors, packages, and dependency relationships

Analysis     -> explores the ecosystem through Cypher and notebooks

This structure keeps the project easy to inspect, rerun, debug, and extend.

Data Collection Layer

The data collection layer is the first major technical component of PHP Packages Graph.

Its role is to transform Packagist from a public package registry into a local research dataset. Packagist exposes package names, package metadata, versions, dependencies, downloads, repository information, maintainers, licenses, and package types.

The collector was designed around one core idea:

Data collection should be reproducible, resumable, and separate from graph import.

Instead of mixing API calls directly with Neo4j writes, PHP Packages Graph first creates a local JSON dataset. This makes the pipeline easier to inspect, debug, rerun, and extend.

The collection workflow is simple:

This separation matters because fetching ecosystem-scale data is slow, fragile, and exposed to network failures.

A graph import should not fail just because one HTTP request failed. A research notebook should not depend on live API calls. A local dataset gives the project a stable intermediate layer.

Package List Collection

The first step fetches the global package list from Packagist:

/packages/list.json

This produces package names in the standard Composer format:

vendor/package

That format becomes the base identifier for the rest of the system.

The collector then uses each package name to fetch detailed metadata:

/packages/{vendor}/{package}.json

Each package is saved under a path that mirrors its Composer identity:

packages/vendor/package.json

That is a small but useful design decision. The filesystem structure follows the package naming structure, which makes the dataset easier to navigate manually.

Incremental Fetching

The collector checks whether package information already exists before fetching it again.

That gives the pipeline a basic incremental behavior:

already fetched -> skip

missing package -> fetch

force update    -> refetch

This matters because Packagist contains a large number of packages, and refetching everything every time would waste time and bandwidth.

The current version keeps the strategy simple. It uses request timeouts, local existence checks, and command-line flags:

--fetch-list

--fetch-info

--force-update

That is enough for a research pipeline.

Why the Collection Design Matters

The important engineering decision is that collection is not analysis.

The collector does not try to answer research questions. It creates a reliable dataset boundary.

That makes the rest of the project cleaner:

Collector -> owns HTTP and local JSON files

Models    -> own validation and normalization

Importer  -> owns graph creation

Queries   -> own analysis

This is better than building one large script that downloads packages, parses JSON, writes nodes, creates relationships, and prints charts in the same execution path.

Graph Database Layer

Once the package metadata is collected, the next challenge is representation.

A relational table can store package rows. A document store can store package JSON. But dependency ecosystems are relationship-heavy by nature.

That makes Neo4j a strong fit for this project.

The graph database was designed around one core idea:

Model the PHP ecosystem as a network of relationships, not as a flat list of packages.

At the center of the graph is the Package node.

A package belongs to a vendor and connects to other packages through Composer dependency relationships:

This model makes the ecosystem queryable in the same shape as the real dependency domain.

A package is not only a record. It is a node in a dependency network.

Technology Choice

PHP Packages Graph uses Neo4j as the graph database.

The project runs Neo4j through Docker Compose and enables APOC and Graph Data Science plugins, which are useful foundations for graph-oriented analysis and future centrality algorithms.

The local Neo4j service exposes:

7474 -> Neo4j Browser

7687 -> Bolt protocol

This gives the project two interfaces:

Neo4j Browser -> visual exploration and manual Cypher queries

Python driver -> scripted graph import and updates

That split works well for a research project. Python handles data processing, while Neo4j handles graph storage and interactive analysis.

Core Graph Model

The first graph import creates two node types:

Vendor

Package

And one ownership relationship:

(v:Vendor)-[:OWNS]->(p:Package)

The package node stores the canonical Composer package name:

full_name: vendor/package

This property becomes the main lookup key for later enrichment and dependency mapping.

The importer also creates uniqueness constraints:

Vendor.name

Package.full_name

That matters because graph imports should be safe to rerun.

Without uniqueness constraints, the database could silently create duplicate vendors or duplicate package nodes. With constraints and MERGE, imports become more predictable and idempotent.

Graph Schema

The graph schema can be represented as a compact ecosystem model:

This schema keeps the model simple while preserving the relationships needed for ecosystem analysis.

Package as a Canonical Knowledge Object

After the initial nodes are created, the mapping layer enriches each package node with metadata:

description

published_at

updated_at

licenses

versions

authors

repository

github_stars

github_watchers

github_forks

github_open_issues

language

abandoned

downloads

type

has_stable_release

is_custom_type

This turns each Package node into a compact knowledge object.

The graph does not only know that a package exists. It can also store what kind of package it is, whether it has stable releases, which licenses appear across versions, how many downloads it has, when it was published, when it was last updated, and whether it has been abandoned.

That combination is what makes the graph useful for research.

Dependency relationships explain structure. Package properties explain context.

Modeling and Normalization Layer

Packagist package metadata is nested, versioned, and inconsistent across packages.

A package can have many versions. Each version can define its own dependencies, licenses, authors, type, stability, and replacement rules.

The modeling layer exists to turn that version-level complexity into package-level signals.

The model was designed around one core idea:

Preserve version-level metadata, but expose package-level signals for graph analysis.

PHP Packages Graph uses Pydantic models to validate and structure package data before importing it into Neo4j.

The model includes package-level objects such as:

Package

Downloads

Maintainer

Author

Version

The Version model captures Composer fields such as:

require

require_dev

suggest

conflict

provide

replace

license

authors

version

version_normalized

abandoned

The Package model then aggregates these fields across all versions.

Aggregated Package Signals

The model computes package-level values such as:

aggregate_versions()

aggregate_licenses()

aggregate_authors()

aggregate_require()

aggregate_require_dev()

aggregate_suggest()

aggregate_conflict()

aggregate_provide()

aggregate_replace()

has_stable_version()

last_updated_time()

is_custom_type()

This is where the dataset becomes analyzable.

Instead of asking every Cypher query to inspect every version object, the Python layer prepares useful package-level properties and relationship lists before writing to Neo4j.

That keeps the graph clean.

Neo4j stores the relationships. Python handles the messy normalization.

Dependency Relationship Mapping

The dependency mapping layer creates explicit graph relationships from Composer metadata:

REQUIRES

DEV_REQUIRES

CONFLICTS

PROVIDES

REPLACES

SUGGESTS

This distinction matters.

A production dependency is not the same as a development dependency. A package that is frequently required in require-dev may be central to testing, static analysis, coding standards, debugging, or development tooling, but not necessarily runtime execution.

By keeping these relationship types separate, PHP Packages Graph can answer more precise questions:

Which packages are runtime infrastructure?

Which packages are development infrastructure?

Which packages appear in both roles?

Which packages replace or provide other packages?

Which packages are suggested but not required?

That is the kind of question a flat package table cannot answer cleanly.

Why the Modeling Design Matters

The main value of the modeling layer is simplification without losing meaning.

Composer metadata is rich, but raw richness can become noise. A research graph needs the right abstractions.

PHP Packages Graph keeps the important distinctions:

package identity

vendor ownership

runtime dependencies

development dependencies

metadata signals

maintenance signals

licensing signals

stability signals

The result is a graph that is expressive enough for ecosystem analysis but simple enough to query directly.

Analysis Layer

Once the graph is built, Cypher becomes the research interface.

The analysis layer asks ecosystem-level questions directly against the graph.

The analysis layer was designed around one core idea:

Turn package metadata into ecosystem-level questions.

The project includes Cypher queries for license distribution, top required packages, development dependencies, packages that appear in both dependency categories, downloads, author averages, release years, and long-maintained packages.

Those queries are not just database checks. They represent the research direction of the project.

License Distribution

Licensing is one of the most important ecosystem-level signals.

The graph can answer:

Which licenses are most common?

How many packages use each license?

Which license dominates the ecosystem?

This matters because licenses influence adoption, compliance, redistribution, and organizational risk.

A dependency graph without license analysis only tells part of the story.

Dependency Influence

The most important graph question is package influence.

A package with many incoming REQUIRES relationships is structurally important because many other packages depend on it.

The graph can answer:

Which packages are required the most?

Which packages are required the most in development?

Which packages are important in both runtime and development contexts?

This reveals the infrastructure layer of the PHP ecosystem.

Some packages may not look exciting as products, but they are deeply embedded in the software supply chain. Those packages deserve attention because their maintenance, stability, and security affect many downstream projects.

Downloads vs Dependency Centrality

Downloads and dependency centrality are related, but they are not the same signal.

Downloads measure usage volume.

Incoming dependency relationships measure structural influence.

A package can have high downloads because it is installed directly by many projects. Another package can be structurally important because it sits underneath other popular packages.

PHP Packages Graph makes it possible to compare these signals instead of treating popularity as one-dimensional.

That is important because ecosystem health depends on both visible and invisible infrastructure.

Stability and Maintenance

The project also studies stability and maintenance through fields such as:

has_stable_release

published_at

updated_at

abandoned

versions

These fields help answer questions like:

Which packages have stable releases?

Which packages are old but still maintained?

Which packages appear abandoned?

Which package types are common?

Which package types fall outside Composer’s expected categories?

That turns package metadata into sustainability signals.

A dependency ecosystem is not healthy only because it is large. It is healthy when important packages are maintained, stable, understandable, and legally usable.

Query Examples

The project uses Cypher queries to explore the graph.

For example, license distribution can be queried with:

MATCH (p:Package)

UNWIND p.licenses AS license

RETURN license, COUNT(p) AS package_count

ORDER BY package_count DESC;

Top runtime dependencies can be queried with:

MATCH (:Package)-[:REQUIRES]->(p:Package)

RETURN p.full_name AS package, COUNT(*) AS times_required

ORDER BY times_required DESC

LIMIT 100;

Top development dependencies can be queried with:

MATCH (:Package)-[:DEV_REQUIRES]->(p:Package)

RETURN p.full_name AS package, COUNT(*) AS times_required

ORDER BY times_required DESC

LIMIT 100;

Packages that are important in both runtime and development contexts can be queried with:

MATCH (p:Package)

OPTIONAL MATCH (:Package)-[:REQUIRES]->(p)

WITH p, COUNT(*) AS required_count

OPTIONAL MATCH (:Package)-[:DEV_REQUIRES]->(p)

WITH p, required_count, COUNT(*) AS dev_required_count

WHERE required_count > 0 AND dev_required_count > 0

RETURN p.full_name, required_count, dev_required_count

ORDER BY required_count DESC, dev_required_count DESC

LIMIT 10;

Old maintained packages can be queried with:

MATCH (p:Package)

WHERE (p.abandoned IS NULL OR p.abandoned = false)

  AND p.published_at IS NOT NULL

  AND p.updated_at IS NOT NULL

WITH DISTINCT p,

     duration.between(datetime(p.published_at), datetime(p.updated_at)) AS lifespan

ORDER BY lifespan.years DESC

LIMIT 10

RETURN DISTINCT p.full_name AS package_name, p.published_at, p.updated_at, lifespan.days;

These queries make the graph useful as a research tool, not just a storage backend.

Research Notebook and Figures

The repository also includes a notebook and figures, which makes sense for the project direction.

The codebase is not only an importer. It is also an exploratory research environment.

The notebook can be used to inspect results, build charts, compare metrics, and turn graph queries into visual explanations.

The figures are the communication layer of the research.

A graph database helps ask questions. A notebook helps interpret the answers. Figures help make those answers understandable.

The research workflow looks like this:

That workflow keeps the project grounded in real data while still making the results understandable outside the database.

Key Engineering Decisions

Start with the ecosystem question, not the tool

The project started from a research question:

What does the PHP ecosystem look like when Packagist packages are modeled as a dependency graph?

That question shaped the architecture.

The goal was not “use Neo4j” or “fetch Packagist data.” The goal was to understand dependency structure, package influence, stability, licensing, and maintenance trends.

Neo4j is useful because the problem is graph-shaped.

Use a graph database for dependency relationships

Package ecosystems are networks.

A package can require, suggest, replace, conflict with, or provide another package. These are relationships, not just columns.

Using Neo4j makes these relationships first-class.

That is the right storage model for dependency analysis.

Separate collection from import

The collector writes a local JSON dataset before data enters Neo4j.

That is a strong research decision.

It makes the system easier to rerun, inspect, debug, and extend. It also prevents the graph import from depending on live API availability.

Keep relationship types explicit

The project does not collapse every edge into a generic DEPENDS_ON relationship.

It keeps Composer relationship types separate:

REQUIRES

DEV_REQUIRES

CONFLICTS

PROVIDES

REPLACES

SUGGESTS

That is the correct modeling decision because each relationship means something different.

Runtime dependency influence and development dependency influence should not be mixed too early.

Aggregate package metadata before graph insertion

The Pydantic model aggregates licenses, authors, versions, dependencies, stability, and update timestamps before writing to Neo4j.

That keeps the graph simpler and the queries cleaner.

The database should not have to understand every nested detail of Packagist JSON just to answer basic ecosystem questions.

Use Cypher as the research interface

Cypher is a good fit because the research questions are relationship-oriented.

Queries like “top required packages” or “packages required both at runtime and in development” are natural graph queries.

That makes the database more than persistence. It becomes an analytical interface.

Prepare for graph algorithms

The Neo4j setup includes the Graph Data Science plugin, which creates room for deeper analysis later.

The natural next step is to compute:

degree centrality

PageRank

betweenness centrality

community detection

dependency clusters

critical package paths

That would push the project from descriptive analysis toward structural ecosystem research.

Treat maintenance as an ecosystem signal

The project does not only look at downloads or dependency counts.

It also includes fields such as abandoned status, publication date, update date, stable release presence, versions, and package type.

That is important because dependency health is not only about popularity.

A heavily depended-on package that is abandoned is a risk. An old package that is still maintained is infrastructure. A package with no stable release may be useful, but it carries a different kind of adoption signal.

What I Learned

The biggest lesson from PHP Packages Graph is that a software ecosystem is not just a collection of packages.

It is an infrastructure network.

Some packages are visible because developers install them directly. Others are invisible because they sit deep in the dependency tree. But both can be important.

Graph modeling makes that structure visible.

I also learned that package metadata becomes more valuable when it is connected. A license field is useful. A download count is useful. An abandoned flag is useful. But the real insight comes from combining those signals with dependency relationships.

For example, an abandoned package with few dependents is one kind of problem. An abandoned package with thousands of incoming dependency relationships is a very different problem.

The project also showed that ecosystem research needs careful data preparation. Packagist metadata is rich, but it is nested across versions and not always uniform. Before analysis can happen, the data needs to be validated, normalized, aggregated, and mapped into the right structure.

Finally, Neo4j is not just a visualization tool here. It changes the kinds of questions the project can ask. Once packages become nodes and dependencies become edges, the PHP ecosystem can be studied as a network.

Conclusion

PHP Packages Graph is a graph-based research system for analyzing the PHP package ecosystem.

Its main engineering value is not that it fetches package metadata from Packagist. The value is the pipeline around that metadata: collection, local dataset creation, validation, normalization, graph modeling, dependency mapping, Cypher analysis, and research visualization.

The system turns Packagist package data into a connected graph where packages can be analyzed by influence, dependency role, license, stability, downloads, age, maintenance, and relationship structure.

That is the core idea behind PHP Packages Graph:

Make the hidden structure of the PHP ecosystem visible, queryable, and analyzable.