A quick research overview of the Research Handbook on Cyberwarfare

Introduction

As you were reading the latest Research Handbook on Cyberwarfare (edited by Tim Stevens and Joe Devanny) you probably felt, like I did, that the same authors were being cited over and over again. Then, like me, you probably did a bunch of work loading the paper citations into a graph database to see if those intuitions were right. 

Findings: Each paper is connected to a lot of citation authors, and only a few (by percentage) are shared amongst the group. (click image to expand)

Beyond understanding how wrong I was about the community, it's sometimes nice to get a bit of meta-data about the cyber policy research community, using this Handbook as a little snapshot. Once the data is parsed and loaded up, you can easily identify central figures, evaluate the research community's general cohesion, and examine the interconnectedness of different thematic areas. 

Methodology

The basic methodology was to take each paper in PDF form, parse it into Text, then have gpt4-o-mini (at a cost of $1.50 USD) parse the citations section and collect a list of last names and first initials, which we then use to form a directed graph. We then can use standard graph algorithms to provide insights on the network of research presented by the Handbook as a whole.

We derive statistics from this graph in two ways: Once with the whole graph, but then again with only citation authors that are shared by at least two papers in the Handbook. As you can see from the visualization above, most authors are only cited by one paper! This graph structure would unnecessarily inflate the community sizes for authors that were unusually persistent in providing citations at every turn. Deriving a subgraph that only used shared cited-authors provides a cleaner analysis for reflecting the most significant connections between authors in the community.

Centrality Measures

We calculated several centrality measures to identify key authors and assess their influence within both the original and refined networks:

• In-Degree Centrality: Measures the number of citations an author receives, identifying the most influential authors within the network.
• Out-Degree Centrality (Original Network): Measures the total number of citations an author makes, which can indicate comprehensive engagement with the literature but may also reflect a tendency to cite broadly, including many sources that are peripheral to the core discussions.
• Out-Degree Centrality (Refined Network): Measures citations made to other non-unique nodes, focusing on an author’s engagement with widely recognized research, thereby filtering out the noise from peripheral citations.
• Betweenness Centrality: Measures the extent to which an author serves as a bridge between different parts of the network, identifying those with the widest spread of influence.

Community Detection and Analysis

Community detection was performed using the Louvain method, which identifies groups of nodes that are more densely connected internally than with the rest of the network. Each community was named based on the most central author within that community, as identified by their degree centrality.

In addition to detecting communities, we analyzed the modularity of the network to assess the strength of the community structure. Modularity is a measure of the extent to which the network can be divided into clearly defined communities, with higher values indicating stronger community structure.

Inter-Community Edge Analysis

To understand the relationships between different research areas, we analyzed the edges connecting different communities. This analysis reveals the level of interaction between distinct research clusters and helps identify areas of overlap or interdisciplinary collaboration.

Results

General Network Overview

After excluding unique citations, the refined network consisted of 257 nodes and 784 edges, representing a focused and well-connected citation network. This refinement resulted in a more accurate depiction of the core contributors and the primary thematic areas within the field of cyber warfare.

Key Contributors

Top Authors by In-Degree (Most Cited):

• Stevens, T. (16 citations) (The editor of the Handbook)
• Smeets, M. (14 citations)
• Rid, T. (13 citations)
• Greenberg, A. (11 citations) (A journalist!)
• Borghard, E. D. (10 citations) (actually slightly higher since a couple citations went to slightly different name)

These authors are recognized as central figures within the network, frequently cited by others. Their work forms the foundation of the academic discourse in cyber warfare, indicating their significant influence on the field.

Comparison of Out-Degree Across Networks:

Top Authors by Out-Degree (Original Network):

1. J.D. Work: 170 citations made
   - Paper: "Offensive cyber capabilities"
2. Gil Baram and Noya Peer: 145 citations made
   - Paper:"Cyberwarfare norms and the attribution imperative: shaping responsible state behaviour in cyberspace"
3. Jamie MacColl and Tim Stevens: 139 citations made
   - Paper: "Countering non-state actors in cyberspace"
4. Simon Henderson: 138 citations made
   - Paper: "Deception in cyberwarfare"
5. Nadiya Kostyuk and Jen Sidorova: 135 citations made
   - Paper: "Military cybercapacity: measures, drivers and effects"

In the original network, authors like J.D. Work and Gil Baram and Noya Peer stood out for their extensive citations, indicating a broad engagement with the literature. However, this high out-degree might reflect a strategy of citing a wide array of sources, some of which could be peripheral or less central to the field's main discussions.

Top Authors by Out-Degree (Refined Network):

1. Gil Baram and Noya Peer: 62 citations made
   - Paper: "Cyberwarfare norms and the attribution imperative: shaping responsible state behaviour in cyberspace"
2. Stéphane Taillat: 61 citations made
   - Paper: "Conceptualizing cyberwarfare"
3. Jamie MacColl and Tim Stevens: 55 citations made
   - Paper: "Countering non-state actors in cyberspace"
4. Miguel Alberto Gomez, Grace B. Mueller, Ryan Shandler: 49 citations made
   - Paper: "Cyberwarfare research methods"
5. Nadiya Kostyuk and Jen Sidorova: 49 citations made
   - Paper: "Military cybercapacity: measures, drivers and effects"

In the refined network, which excludes unique citations, Gil Baram and Noya Peer, along with Stéphane Taillat, still emerge as central figures. This suggests that their work not only engages broadly but also cites sources that are recognized and referenced by others within the field, indicating a more integrated approach to scholarship.

Community Structure

The community detection analysis revealed several key research clusters on the refined network, each named after the most central author and their key paper:

1. Miguel Alberto Gomez, Grace B. Mueller, Ryan Shandler: Cyberwarfare research methods (47 members)
2. Stéphane Taillat: Conceptualizing cyberwarfare (45 members)
3. Jamie MacColl and Tim Stevens: Countering non-state actors in cyberspace (41 members)
4. J.D. Work: Offensive cyber capabilities (39 members)
5. Sean Lawson: War by any other name: a short history of the idea of cyberwarfare in the United States (35 members)

These communities represent the major thematic areas within the field, each with a substantial citation field. The community sizes are relatively balanced, indicating that there are multiple active research areas within the network (as represented by the Handbook as a survey document).

Modularity of the Partitioning:

Modularity Score: 0.3206

The moderate modularity score suggests that while there are distinct communities within the network, there is also significant overlap and interaction between these communities. This reflects a field that is both specialized and interdisciplinary, with research areas that are interconnected (at least, as viewed in the snapshot that is the Handbook). 

Average Degree Centrality per Community:

The communities with the highest average degree centrality were led by Miguel Alberto Gomez, Grace B. Mueller, Ryan Shandler (Cyberwarfare research methods), Gil Baram and Noya Peer (Cyberwarfare norms), and Jamie MacColl and Tim Stevens (Countering non-state actors). These communities are characterized by dense internal connections, indicating strong intra-community collaboration and citation practices. Or, in visual terms, these papers are highly connected to the central community of citations (as you can see from the subgraphs below). 

Gil Baram and Noya Peer

Jamie MacColl and Tim Stevens 

Miguel Alberto Gomez, Grace B. Mueller, Ryan Shandler

Inter-Community Relationships

I like to sometimes wonder which academics are friends with which other academics - or in more graph analysis terms - who is citing the same set of people as determined by their inter-community relationships. This can be driven both by people doing good literature review in their field, and writing papers in related fields, or by people simply all citing their friends.

The inter-community edge analysis revealed significant connections between the largest communities:

1. Stéphane Taillat: Conceptualizing cyberwarfare and Miguel Alberto Gomez, Grace B. Mueller, Ryan Shandler: Cyberwarfare research methods (21 edges)
2. Jamie MacColl and Tim Stevens: Countering non-state actors in cyberspace and Miguel Alberto Gomez, Grace B. Mueller, Ryan Shandler: Cyberwarfare research methods (21 edges)
3. Gil Baram and Noya Peer: Cyberwarfare norms and Jamie MacColl and Tim Stevens: Countering non-state actors in cyberspace (18 edges)

It makes sense that Conceptualizing cyberwarfare and looking at research methods in general are highly congruent. It makes less sense that looking at non-state actors in cyberspace is so highly connected in the citation network to two other unrelated papers.

Discussion and Conclusion

This study provides a comprehensive analysis of the citation network within the field of cyber warfare, highlighting the most influential authors, key research clusters, and the interconnectedness of different research areas, as measured by the snapshot the Research Handbook on Cyberware provides. The exclusion of unique citations allowed for a more focused and accurate representation of the core contributors and their relationships. The initial thesis (that papers were citing all the same people) was incorrect, although it will be interesting to see how these networks compare to next year's Handbook. 

The findings indicate that while the field of cyber warfare is composed of distinct research areas, there is also considerable overlap and collaboration between these areas. The moderate modularity score suggests that the field is not rigidly divided but rather interconnected, with various subfields contributing to a cohesive body of knowledge. It's telling that there are no large communities in the refined network that encompass multiple sets of papers. In other words, the subjective feeling that there are "camps" of cyber policy academics does not play out in the data. That said, the mere existence of a citation does not mean it is taking into account in the argument of a paper. It may be mentioned for performative reasons, and then dismissed textually (something we don't look at here).

The centrality measures identified key figures who play a crucial role in shaping the academic discourse, both by contributing foundational research and by integrating diverse perspectives across the field. The community structure analysis further emphasized the importance of these contributors, showing how their work forms the backbone of several major research areas.

Overall, this analysis provides some fun insights into the structure and dynamics of current cyber warfare research, offering a foundation for future studies to build upon. Understanding these patterns can help guide researchers and practitioners as they navigate the complex and evolving landscape of cyber warfare academic research, for fun and profit.