Examined Line: The Art of Source Code Analysis with Sokrates

Background: Sokrates Philosophy

2020-05-25T20:12:01+02:00

“Socrates believed that philosophy – the love of wisdom – was the most important pursuit above all else. For some, he exemplifies more than anyone else in history the pursuit of wisdom through questioning and logical argument, by examining and by thinking. His ‘examination’ of life in this way spilled out into the lives of others, such that they began their own ‘examination’ of life, but he knew they would all die one day, as saying that a life without philosophy – an ‘unexamined’ life – was not worth living.”

Sokrates is a tool that can help you to understand your code by making visible the size, complexity, and coupling of software. With Sokrates, my goal was to create a tool that ordinary software developers and software architects could use in daily work, regardless of how big or small their project is, irrespective of technology.

My primary motivation for building Sokrates was dealing with issues in my daily practice. I am always surprised how difficult it can be to get quick answers to basic, simple questions about your source code:

How much code do you here?
How much of it you’ve written yourself?
How much of it do you maintain?
What part of your code has not changed recently?
Which programming languages do you use?
What is your primary or main code, the one used to define the structure and logic of the system in production?
What is your secondary code, the one needed to test and build your system, but itself is not running in production?
Do you generate some part of your codebase? How?
How duplicated is your codebase? Why?
How big are your files?
How many units do you have? How big are they? How complex?
How files in your codebase depend on each other?

To answer these and many other questions, Sokrates offers tools, which we will discuss later in the book.

Sokrates: A Practical Tool vs. An Experiment

Sokrates implements my vision on how to document and analyze software architectures of software systems. Sokrates is an experimental tool that also explores how far following a simple approach to building code analysis tools can get you. In creating Sokrates, I also wanted to prove that it is possible to develop a practical tool but without all complexity frequently associated with static code analysis tools.

The result of my experiment is dual. First, I think I have managed to build a pretty useful and elegant tool. At least for my daily work, it provides a handy addition. But I have also learned a lot, and with this book, I want to share the lessons learned and hopefully influence new generations of practical and straightforward software analysis tools.

This book describes what I have learned during my work and building Sokrates. It is, in many aspects, subjective, presenting my view, experiences, and ideas. Nevertheless, I do think that you can learn something from it. Most importantly, all that I will describe has been validated in practice in a complex, messy reality.

The journey to this book was through building Sokrates software. But the book and the software are the two sides of the same coin. On the one hand, as I was using Sokrates in my daily work, I made notes for this book. On the other hand, while reflecting on what I wrote, I also started to change some parts of Sokrates based on what I have learned.

Figure 1: The Sokrates project consists of a book and an open-source software tool. They are the two sides of the same coin. Developing software enabled me to implement ideas, test them in practice, and build a useful, proven tool. Writing a book helped me organize ideas, obtain new insights, improve the software, and share the lessons learned.

Design Principles

“Anyone who holds a true opinion without understanding is like a blind man on the right road.” Sokrates

“Give a man a fish and you feed him for a day; teach a man to fish and you feed him for a lifetime.”

Sokrates follows the following design principles:

Teach you how to fish. With the Sokrates project, I do not want only to promote a software tool. I want to teach you how to build similar tools yourself. Whether you decided to use Sokrates or only its ideas, I believe that understanding how the tool work will help you to understand the analysis results better and to use them better to make decisions.
Transparency. Sokrates’ analyses contain no magic. Sokrates reports explicitly explain why some values are there and give you lists of files to double-check the results.
Technology independence. Sokrates is technology independent; in essence, any textual representation could be analyzed at least to some extend by Sokrates. However, Sokrates does offer helpers for accelerating configuration and analysis of standard technologies.
Using Metrics to Start a Discussion, Not to Kill It. Sokrates gives you insights into your code, and some pointer to probe further, but not advice that you can follow blindly.

Sokrates Pragmatic Approach

Sokrates is a practical source code analysis tool. Sokrates does not aim at replicating the level of details of many other tools, such as SonarQube. Instead, it leans on a few basic principles and techniques to get insights about your code without building a massive and complex tool.

The first thing I have learned pretty fast when building Sokrates is that it is extremely tough to build a proper fully-featured parser for any programming language. Any software analysis tool needs a handful of such parsers. Contemporary software projects frequently include a dozen or more of different programming languages and dialects.

So how have I solved this problem? By not building fully-featured parsers.

I’ve also learned when building Sokrates that you can go a long way with using a few simple, heuristic parsing techniques. You may lose details in the process, but if you focus on the essence, such simple heuristic techniques can offer you incredible value for money. In many instances, you may get the same details as more complex and expensive tools. And performance-wise, simple heuristic techniques often work order of magnitude faster.

In the following chapters, we outline a few of these techniques. The following is useful for those who want to contribute or use the Sokrates source code. But it may also offer few insights into power and limitation of Sokrates analyses.

Background

Sokrates looks on the source code from a perspective of maintenance, making visible the size, complexity and coupling of software.

For more details see my O’Reilly Video Training (from my time at Software Improvement Group): Building Maintainable Software (4 hours), and O’Reilly Webcast: Building Maintainable Software (1 hour, together with Rob van der Leek).

The fragment of my training video on building maintainable software is freely avaliable at Youtube:

Appendix: Command Line Interface (CLI)

2020-05-25T20:12:01+02:00

Sokrates command-line interface (CLI) includes five commands:

init,
generateReports,
updateConfig,
initLandscape, and
updateLandscape.

init command

The init command generates a new Sokrates configuration file based on standard source code conventions.

java -jar sokrates-LATEST.jar init [options]
   -srcRoot  <arg>   [OPTIONAL] the path to source code root folder (default is <currentFolder>)
   -confFile <arg>   [OPTIONAL] the path to configuration file (default is "<currentFolder>/_sokrates/config.json")

Figure 1: The init command generates a new Sokrates configuration file based on standard source code conventions.

generateReports command

The generatesReports command generates HTML reports and exports data files.

  java -jar sokrates-LATEST.jar generateReports [options]
         -confFile     <arg>           [OPTIONAL] the path to configuration file (default is "<currentFolder>/_sokrates/config.json")
         -outputFolder <arg>           [OPTIONAL] the folder where reports will be stored (default is "<currentFolder>/_sokrates/reports/")
         -reportAll                    [DEFAULT] generate all reports
         -reportData                   save only data in JSON and text formats (stored in <outputFolder>/data)
         -reportOverview               generate only the report describing the overview of files in scope (stored in <outputFolder>/html/SourceCodeOverview.html)
         -reportDuplication            generate only the duplication report (stored in <outputFolder>/html/Duplication.html)
         -reportLogicalDecomposition   generate only the components logical decomposition report (stored in <outputFolder>/LogicalDecomposition.html)
         -reportFileSize               generate only the file size report (stored in <outputFolder>/html/FileSize.html)
         -reportUnitSize               generate only the unit size report (stored in <outputFolder>/html/UnitSize.html)
         -reportConditionalComplexity  generate only the conditional complexity report (stored in <outputFolder>/html/ConditionalComplexity.html)
         -reportConcerns               generate only the concerns report (stored in <outputFolder>/html/Concerns.html)
         -reportMetrics                generate only the metrics overview report (stored in <outputFolder>/html/Metrics.html)
         -reportControls               generate only the controls report (stored in <outputFolder>/html/Controls.html)
         -reportFindings               generate only the findings report (stored in <outputFolder>/html/Findings.html)
         -internalGraphviz             use internal Graphviz library (by default external dot program is used, you may specify
                                       the external dot path via the system variable GRAPHVIZ_DOT)

Figure 2: The generatesReports command generates HTML reports and exports data files.

updateConfig command

The updateConfig updates a configuration file and completes missing fields. You can also use the updateConfig command to upgrade your configuration file to the latest version.

java -jar sokrates.jar updateConfig [options]
     -confFile <arg>         [OPTIONAL] the path to configuration file (default is "<currentFolder>/_sokrates/config.json"
     -setDescription <arg>   [OPTIONAL] sets a project description
     -setLogoLink <arg>      [OPTIONAL] sets a project logo link
     -setName <arg>          [OPTIONAL] sets a project name
     -skipComplexAnalyses    [OPTIONAL] skips complex analyses (duplication, dependencies, file caching)

Figure 3: The updateConfig updates a configuration file and completes missing fields.

initLandscape command

The initLandscape creates a Sokrates Landscape report and an initial landscape configuration file for landscape analyses.

java -jar sokrates.jar initLandscape [options]
     -analysisRoot <arg>    the path to the root folder of multiple Sokrates projects, scans all subfolders for Sokrates reports and configurations.  The folder where landscape reports will be stored (default is "<currentFolder>/_sokrates_landscape/reports/".
     -confFile     <arg>    [OPTIONAL] the path to landscape configuration file (default is "<currentFolder>/_sokrates_landscape/config.json"

Figure 4: The initLandscape creates a Sokrates Landscape report and an initial landscape configuration file for landscape analyses.

updateLandscape command

The updateLandscape updates a Sokrates Landscape report, adding new and removing non-existent projects without changing other settings in the landscape configuration file.

java -jar sokrates.jar updateLandscape [options]
     -analysisRoot <arg>   the path to the root folder of multiple Sokrates projects, scans all subfolders for Sokrates reports and configurations. The folder where landscape reports will be stored (default is "<currentFolder>/_sokrates_landscape/reports/".
     -confFile     <arg>   [OPTIONAL] the path to landscape configuration file (default is "<currentFolder>/_sokrates_landscape/config.json"

Figure 5: The updateLandscape updates a Sokrates Landscape report, adding new and removing non-existent projects without changing other settings in the landscape configuration file.

Example: A Getting Started Video

See a 5 minutes video on using Sokrates CLI to analyze source code of Sokrates:

Example: Running Sokrates as a Batch Process

The following bash script illustrates a more complex scenario of using multiple commands of the Sokrates CLI.

# remove the temporary analysis folder
rm -rf analysis

# remove the reports folder
rm -rf reports

# create temporary analysis folder
mkdir analysis
mkdir analysis/_sokrates
#create the reports folder
mkdir reports

# copy Sokrates configuration file to the temporary folder
cp config.json analysis/_sokrates

cd analysis

# install the latest version of Sokrates
curl -O https://d2bb1mtyn3kglb.cloudfront.net/builds/sokrates-LATEST.jar

# checkout the code
git clone https://github.com/zeljkoobrenovic/sokrates.git
mv temp-git-dir/* .
rm -rf temp-git-dir

# run analysis
java -jar -Xmx10g sokrates-LATEST.jar generateReports -reportAll

# copy generated reports to the reports folder
cp -r _sokrates/reports/* ../reports

cd ..
# remove the temporary analysis folder
rm -rf analysis

Figure 5: A bash script illustrating a more complex scenario of the Sokrates command-line interface.

Appendix: Sokrates Configuration File

2020-05-25T20:12:01+02:00

Configuration File for Individual Projects

Sokrates configuration is defined in a JSON file (see an example here).

You can use the init command to generate default file for your project.

The default location of the configuration file is /_sokrates/config.json

Based on this configuration Sokrates will generate a number of reports. The default reports folders is /_sokrates/reports/

Figure 1: The Sokrates’ configuration file describes how Sokrates should view and analyze the source code..

The configuration file has several sections, you can use to:

describe your project with a name, logo, summary text, and to add external links
define the scope, including the location of the source code root, file extensions and files to analyze, ignored files, files classification, defining what is main code, test code, generated code, build & deployment code, or other code…
identify logical decompositions: one or more ways to looks at components in the system
define concerns: aspects of a software system that cannot be cleanly decomposed from the rest of the system
set goals and controls: define measurements and alarms to keep your system within the desired values
provide analysis reference points: for comparison and trend analysis
add summary findings: manually added one-line insights and notes
perform advanced configuration: meta-rules for components, concerns and dependencies

Description of the project

Describes your project with a name, logo, summary text, and to add external links.

"metadata":{
  "name":"Sokrates",
  "description":"A pragmatic source code analyzer.",
  "logoLink":"https://sokrat.org/assets/logo.png",
  "links":[
     {
        "label":"sokrat.org",
        "href":"https://sokrat.org/"
     }
  ]
}

Define the scope

Defines the scope, including the location of the source code root, file extensions and files to analyze, ignored files, files classification, defining what is main code, test code, generated code, build & deployment code, or other code.

{
"srcRoot":"..",
"extensions":[
  "java",
  "html",
  "xml",
  "md",
  "txt"
],
"ignore":[
  {
     "pathPattern":".*/node_modules/.*",
     "contentPattern":"",
     "include":true,
     "note":""
  },
  {
     "pathPattern":".*/target/.*",
     "contentPattern":"",
     "include":true,
     "note":"Compiled files"
  }
]
}

For analysis purposes Sokrates separate files in scope into several categories: main, test, generated, deployment and build, and other.
The main category contains all manually created source code files that are being used in the production.
Files in the main category are used as input for other analyses: logical decomposition, concerns, duplication, file size, unit size, and conditional complexity.
Test source code files are used only for testing of the product. These files are normally not deployed to production.
Build and deployment source code files are used to configure or support build and deployment process.
Generated source code files are automatically generated files that have not been manually changed after generation.
While a source code folder may contain a number of files, Sokrates is primarily interested in the source code files that are being written and maintained by developers.
Files containing binaries, documentation, or third-party libraries, for instance, are excluded from analysis. The exception are third-party libraries that have been changed by developers.

{
    "main": {
        "name": "main",
        "sourceFileFilters": [
            {
                "pathPattern": ".*",
                "contentPattern": "",
                "exception": false,
                "note": ""
            }
        ],
        "files": []
    },
    "test": {
        "name": "test",
        "sourceFileFilters": [
            {
                "pathPattern": ".*/[Tt]est/.*",
                "contentPattern": "",
                "exception": false,
                "note": "Test files"
            }
        ],
        "files": []
    },
    "generated": {
        "name": "generated",
        "sourceFileFilters": [],
        "files": []
    },
    "buildAndDeployment": {
        "name": "build and deployment",
        "sourceFileFilters": [],
        "files": []
    },
    "other": {
        "name": "other",
        "sourceFileFilters": [],
        "files": []
    }
}

Define logical decompositions: one or more ways to looks at components in the system

Identifies logical decompositions: one or more ways to looks at components in the system.

{
   "logicalDecompositions":[
      {
         "name":"primary",
         "scope":"main",
         "filters":[

         ],
         "componentsFolderDepth":0,
         "components":[
            {
               "name":"Logging",
               "sourceFileFilters":[
                  {
                     "pathPattern":".*/LoggingService/.*",
                     "contentPattern":"",
                     "exception":false,
                     "note":""
                  },
                  {
                     "pathPattern":".*/LogQueueProbe/.*",
                     "contentPattern":"",
                     "exception":false,
                     "note":""
                  },
                  {
                     "pathPattern":".*/Logging/.*",
                     "contentPattern":"",
                     "exception":false,
                     "note":""
                  }
               ],
               "files":[

               ]
            }
         ],
         "metaComponents":[

         ],
         "includeRemainingFiles":true,
         "dependenciesFinder":{
            "useBuiltInDependencyFinders":true,
            "rules":[

            ],
            "metaRules":[

            ]
         },
         "renderingOptions":{
            "orientation":"TB"
         },
         "includeExternalComponents":true,
         "duplicationLinkThreshold":50
      }
   ]
}

Define concerns: aspects of a software system that cannot be cleanly decomposed from the rest of the system

Defines concerns: aspects of a software system that cannot be cleanly decomposed from the rest of the system.

{
   "concernGroups":[
      {
         "name":"general",
         "concerns":[
            {
               "name":"exception handling",
               "sourceFileFilters":[
                  {
                     "pathPattern":".*[.]java",
                     "contentPattern":".* try \\{.*",
                     "include":true,
                     "note":""
                  }
               ]
            },
            {
               "name":"logging",
               "sourceFileFilters":[
                  {
                     "pathPattern":"",
                     "contentPattern":"import .*logging[.]Log.*",
                     "include":true,
                     "note":""
                  }
               ]
            },
            {
               "name":"file operations",
               "sourceFileFilters":[
                  {
                     "pathPattern":"",
                     "contentPattern":"import .*java[.]io[.]File.*",
                     "include":true,
                     "note":""
                  }
               ]
            },
            {
               "name":"javafx",
               "sourceFileFilters":[
                  {
                     "pathPattern":"",
                     "contentPattern":"import javafx.*",
                     "include":true,
                     "note":""
                  }
               ]
            }
         ]
      }
   ]
}

Set goals and controls: define measurements and alarms to keep your system within the desired values

Sets goals and controls: define measurements and alarms to keep your system within the desired values.

{
    "goalsAndControls": [
        {
            "goal": "Keep the system simple and easy to change",
            "description": "Aim at keeping the system size modest (less than 200,000 LOC is good), duplication low (less than 5% is good), files small (no files longer than 1000 LOC is good), and units simple (no units with more than 25 decision points is good).",
            "controls": [
                {
                    "metric": "LINES_OF_CODE_MAIN",
                    "description": "Total number of lines of main code",
                    "desiredRange": {
                        "min": "0",
                        "max": "200000",
                        "tolerance": "20000"
                    }
                },
                {
                    "metric": "DUPLICATION_PERCENTAGE",
                    "description": "System duplication",
                    "desiredRange": {
                        "min": "0",
                        "max": "5",
                        "tolerance": "1"
                    }
                },
                {
                    "metric": "NUMBER_OF_FILES_1001_PLUS",
                    "description": "The number of very large files",
                    "desiredRange": {
                        "min": "0",
                        "max": "0",
                        "tolerance": "1"
                    }
                },
                {
                    "metric": "CONDITIONAL_COMPLEXITY_DISTRIBUTION_26_PLUS_COUNT",
                    "description": "Number of very complex units",
                    "desiredRange": {
                        "min": "0",
                        "max": "0",
                        "tolerance": "1"
                    }
                }
            ]
        }
    ]
}

Provide analysis reference points: for comparison and trend analysis

Controls enable you to set alarms for any of the Sokrates metrics. An alarm is defined with a desired range and tolerance.

{
   "compareResultsWith": [
     {
        "label": "reference",
        "analysisResultsPath": "history/start/analysisResults.json"
    }
  ]
}

Add summary findings: manually added one-line insights and notes

Optional few bullets to summarize the systems.

{
    "summaryFindings": [
        "note 1",
        "note 2"
    ]
}

Perform advanced configuration: meta-rules for components, concerns and dependencies

One of the most powerful features of Sokrates is the possibility to use Sokrates String Transformation Language to define meta rules to define components, concerns and dependencies.

A meta rule is a search pattern (content and/or path) combined with the string operations to process found string to get the name that to define a component, concern or a dependency to a component.

Meta rules can be used in logical decompositions (the metaComponents field), dependency finders (the metaRules field), and in concerns (the metaConcerns field)

The following SSTL operations are supported:

extract (regex1, regex2,…)
replace (regex, replaceString)
append (text)
prepend (text)
trim
tolowercase
touppercase

Example (components and dependencies finder with meta rules):

{
    "logicalDecompositions": [
        {
            "name": "package level",
            "scope": "main",
            "filters": [],
            "componentsFolderDepth": 0,
            "components": [],
            "metaComponents": [
                {
                    "pathPattern": ".*[.]java",
                    "contentPattern": "package nl[.]obren[.]sokrates[.].*",
                    "use": "content",
                    "ignoreComments": true,
                    "nameOperations": [
                        {
                            "op": "extract",
                            "params": [
                                "nl[.]obren[.]sokrates[.][a-zA-Z0-9_]+[.][a-zA-Z0-9_]+"
                            ]
                        },
                        {
                            "op": "replace",
                            "params": [
                                "nl[.]obren[.]sokrates[.]",
                                ""
                            ]
                        }
                    ]
                }
            ],
            "includeRemainingFiles": false,
            "dependenciesFinder": {
                "useBuiltInDependencyFinders": false,
                "rules": [],
                "metaRules": [
                    {
                        "pathPattern": ".*[.]java",
                        "contentPattern": "import nl[.]obren[.]sokrates[.].*",
                        "use": "content",
                        "ignoreComments": true,
                        "nameOperations": [
                            {
                                "op": "extract",
                                "params": [
                                    "nl[.]obren[.]sokrates[.][a-zA-Z0-9_]+[.][a-zA-Z0-9_]+"
                                ]
                            },
                            {
                                "op": "replace",
                                "params": [
                                    "nl[.]obren[.]sokrates[.]",
                                    ""
                                ]
                            }
                        ]
                    }
                ]
            },
            "renderingOptions": {
                "orientation": "TB"
            }
        }
    ]
}

Example (concerns with meta rules):

{
    "concernGroups": [
        {
            "name": "java technology",
            "concerns": [ ],
            "metaConcerns": [
                {
                    "pathPattern": "",
                    "contentPattern": "import[ ]+java[.].*",
                    "use": "content",
                    "ignoreComments": false,
                    "nameOperations": [
                        {
                            "op": "extract",
                            "params": [
                                "java[.][a-zA-Z0-9_]+"
                            ]
                        }
                    ]
                }
            ]
        }
    ]
}

Appendix: Installation

2020-05-25T20:12:01+02:00

Installation: JAR files

Command Line Interface

COMMAND LINE JAR DOWNLOAD: sokrates-LATEST.jar (38 MB)

Simple usage example (to analyze Junit4):

git clone https://github.com/junit-team/junit4
cd junit4

java -jar sokrates-LATEST.jar init
java -jar sokrates-LATEST.jar generateReports

open _sokrates/reports/html/index.html

Explorer

DOWNLOAD: sokrates-explorer-LATEST.jar (78 MB)

NOTE: require JavaFX (download it and install it from openjfx.io)

java –module-path $JAVAFX_HOME/lib –add-modules=javafx.controls,javafx.web -jar sokrates-explorer-LATEST.jar

Source Code

Sokrates is an open-source project, released under the GNU Affero General Public License v3.0.

To build a project from source code, go to the root of the project, and run the following command:

mvn clean install

The build will create two jar files:

the command line interface JAR in the cli/target folder
the interactive explorer JAR in the codeexplorer/target folder

Prerequirements:

Java 8+
Maven

Appendix: Sokrates String Transformation Language (SSTL)

2020-05-25T20:12:01+02:00

The Sokrates String Transformation Language (SSTL) is a simple embedded programming language enabling you to transform a phrase extracted from a file path or file content into another string.

You define SSTL scripts as a simple list of commands with parameters, a processing pipeline. Each next command takes as an input the result of the previous command.

SSTL supports the following commands:

extract (regex1)
replace (regex, string)
append (text)
prepend (text)
trim
lowercase
uppercase

Sokrates uses SSTL in several places:

to define dynamic names of components in logical decompositions (the meta components section)
to define dynamic names of concerns (the meta concerns section)
to find components names for dependencies (the meta rules section)

Examples

The following fragment of a Sokrates configuration files uses SSTL to find dependencies of files to org.apache modules:

{
    "pathPattern": ".*[.]java",
    "contentPattern": "import org[.]apache[.].*",
    "use": "content",
    "ignoreComments": true,
    "nameOperations": [
        {
            "op": "extract",
            "params": [
                "import org[.]apache[.][a-zA-Z0-9_]+"
            ]
        },
        {
            "op": "replace",
            "params": [
                "import org[.]apache[.]",
                ""
            ]
        }
    ]
}

This script does the following:

looks for any Java file that contains at least one line starting with “import org.apache.” (matches the regex expression “import org[.]apache[.].*”)
it then uses as an input for pressing the found line (the parameter use is set to “content”)
for each such line Sokrates performs two chained SSTL trafsormations:
- extract the regex pattern “import org[.]apache[.][a-zA-Z0-9_]+”
- remove “import org[.]apache[.]” (replace it with an empty string)

Figure 2 illustrates this processing on two examples.

Figure 1: A fragment of a Sokrates configuration files uses SSTL to find dependencies of files to org.apache modules.

Figure 2: The steps of SSTL processing for two import statements for a script defined in Figure 1.

Appendix: Regular Expressions

2020-05-25T20:12:01+02:00

Jeff Atwood, the co-founder of StackOverflow.com, is a big fan of regular expressions. On a shortlist of his mandatory readings for software engineering is the Regular Expression Cookbook, and this is what he says:

I may be a card carrying member of the “Keep It Simple Stupid” club, but I’m making a meteor sized exception for regular expressions. Written properly, they will save you a tremendous amount of time in string manipulation, and I’ve never run across a project where they didn’t come in handy somewhere.

Once you delve into the world of regular expressions, you may become drunk with the amazing power and potential they have, which results in things like Perl. Remember, absolute power corrupts absolutely. But it also rocks absolutely.

You cannot efficiently use Sokrates if you do not know at least the basics of regular expressions. Sokrates uses regular expressions for two purposes: to filter and classify files, extract and manipulate text, name components, or identify dependencies.

Regular Expressions in Sokrates

Sokrates is all about regular expressions. Sokrates is inspired by grep, a command-line tool for searching text data sets for lines that match a regular expression. Its name comes from the ed command g/re/p (globally search for a regular expression and print matching lines). grep was originally developed for the Unix operating system, but later available for all Unix-like systems and some others.

Sokrates uses regular expressions for two main purposes:

Filter and classify files (see the Scoping section for details), and
Regular expressions as a part of Sokrates String Transformation Language (SSTL). You can use SSTL to define the names of components and identify dependencies among the components.

To Probe Further

Online resources:

Books:

Landscape Analysis Overview

2020-05-25T20:12:01+02:00

The Sokrates landscape analysis aggregates and indexes the Sokrates project analysis reports, providing a centralized and uniform view on multiple Sokrates projects. The analysis is called landscape because it draws a map of all projects you want to view from one place.

The landscape analyses provide a helicopter view on huge code bases. The uniform centralized index page provides an easy way to find and access details of individual Sokrates projects. The landscape analysis also provides an efficient way to aggregate and compare findings from dozens or hundreds of such projects.

To run a Sokrates landscape analysis, you need to get to the Sokrates project reports folder. Sokrates landscape analysis assumes that your projects are somewhere in deeper folders. Sokrates will recursively go through all folders, looking for all configuration and analysis result files. Sokrates will create a hierarchical view if other landscape analyses are on deeper levels. For details on Sokrates landscape commands, visit the command line interface page.

Figure 1: The updateLandscape command goes through all folders to find Sokrates projects, adding them to the index list.

Sokrates landscape analysis provides the following types of insights:

Overview of code volume and trends per programming languages
Overview of projects and commit and contributor trends
Overview of contributors and their recent and historical overview
Team topologies overview, rendering dependencies between people working on the same projects

An alternative to the landscape analysis is to scope a whole codebase as one Sokrates project, defining each sub-project as a component. However, for code bases of dozens of millions of lines of code, such an approach may not be practical enough due to the length of analysis. Instead, you can define many smaller projects and use the landscape analysis to provide a bigger picture.

You can find an example Sokrates landscape report at the sokrates.dev site. In this report, I have aggregated and indexed more that 1,000 Apache Software Foundation (ASF) open-source projects, with more than 100 millions of lines of code.

Trend Analysis

2020-05-25T20:12:01+02:00

Trend analysis shows the difference between values measured at different times, comparing the results of the current analysis with previous reports.

Figure 1: A screenshot from a Sokrates trend report detailing trends per metric.

Trend analysis gives the meaning to measurements through comparison. While it may be difficult to tell if some measured value is good or bad, it may still be possible to say if this value is becoming better or worse. For instance, you always have some level of duplication in code. You may not have the time and resources to refactor the system to remove all duplication. But you usually want to have a trend of reduced duplication. You can at least monitor the duplication level so that your new code does not increase overall code repetition.

For more insights into the value of trend analysis, Sokrates recommends reading the section “Favor tracking trends over absolute numbers” in the article An Appropriate Use of Metrics, (MartinFowler.com):

“Looking at trends provides more interesting information than whether or not a target is met. Working out if a goal is met is easy. The difficult work, and one that management must work with people with the skills to complete is looking at trends to see if they are moving in the desired direction and a fast enough rate. Trends provide leading indicators into the performance that emerges from organizational complexity. It is clearly pointless focusing on the gap in a number when a trend moves further and further away from a desired state.”

Sokrates Snapshots

Sokrates supports trend analysis by saving the snapshots of analysis results and comparing these snapshots with current values. Keeping of snapshots is optional, but if you enable these features, you can choose the saving frequency at three levels: daily, weekly, or monthly.

{
    "trendAnalysis": {
        "saveHistory": true,
        "frequency": "weekly",
        "maxReferencePointsForAnalysis": 20,
        "historyFolder": "history"
    }
}

Figure 2: A fragment of Sokrates configuration file, describing parameters of trend analysis and saving of history snapshots.

Each snapshot is an archived file named analysisResults.zip. It contains two JSON files: analysisResults.json, with all measurements, and config.json, with a copy of the Sokrates configuration at the time of the measurement.

Sokrates stores snapshots files in the _sokrates/history/snapshots folder. Each snapshot file has its own subfolder folder with the following naming conventions:

…/history/snapshots/years-month-day
for daily snapshots (e.g., _sokrates/history/snapshots/2020-01-01)
…/history/snapshots/years-weekInYear
for daily weekly (e.g., _sokrates/history/snapshots/2020-42)
…/history/snapshots/years-month
for monthly snapshots (e.g. _sokrates/history/snapshots/2020-01)

Figure 2: A screenshot from a Sokrates trend report detailing volume and duplication change.

Analyzing Trends Retroactively

If you have just started using Sokrates, you can still get the trend analysis by checking out the code at different moments in the past and running the Sokrates analysis against them to get the snapshots of source code from the past. The following line is the Git command to get your source code as it was on January 1, 2020:

git checkout `git rev-list -n 1 --first-parent --before="2020-01-01 00:00" master`

Figure 3: Git commands to get a code on a specified date.

The following code fragment illustrates a more complex example, where you can get the analysis snapshot of any git repository at any date:

#!/bin/bash

# prepare a temporary folder for Sokrates analysis
rm -rf temp
mkdir temp
cp -r sokrates-config/$1/* temp

cd temp

# clone the repository
git clone $2 temp-git-download
cd temp-git-download
# go back in time (the $3 parameter needs to be provided)
git checkout `git rev-list -n 1 --first-parent --before="$3 00:00" master`
cd ..
mv temp-git-download/* .
rm -rf temp-git-download


cd ..

# run Sokrates analysis
java -jar -Xmx10g sokrates.jar generateReports -reportData -confFile temp/_sokrates/config.json -outputFolder temp/_sokrates/reports -reportAll

# make history folders with appropriate date in the name
mkdir history/$1
mkdir history/$1/$3


# copy analysisResults.json and config.json
cp temp/_sokrates/reports/data/analysisResults.json history/$1/$3
cp temp/_sokrates/reports/data/config.json history/$1/$3

rm -rf temp

# zip snapshot files
cd history/$1/$3
zip analysisResults.zip *.json
rm *.json

Figure 3: A bash script to get the analysis snapshot of a git repository at a specified date.

You can then call this batch file as illustrated in the following example of JUnit4 repository:

#!/bin/bash

rm -rf history

mkdir history
mkdir history/java

bash run-analysis-history.sh 'java/junit4' 'https://github.com/junit-team/junit4' '2020-01-01'
bash run-analysis-history.sh 'java/junit4' 'https://github.com/junit-team/junit4' '2019-01-01'
bash run-analysis-history.sh 'java/junit4' 'https://github.com/junit-team/junit4' '2018-01-01'
bash run-analysis-history.sh 'java/junit4' 'https://github.com/junit-team/junit4' '2017-01-01'
bash run-analysis-history.sh 'java/junit4' 'https://github.com/junit-team/junit4' '2016-01-01'
bash run-analysis-history.sh 'java/junit4' 'https://github.com/junit-team/junit4' '2015-01-01'

Figure 5: A bash script describing the full process of checking our source code on different dates, by calling the script defined in Figure 4.

Controls

2020-05-25T20:12:01+02:00

In Sokrates, you can define traffic lights like controls. For each of the metrics that Sokrates measures, you can specify a control with the threshold and tolerance. Sokrates will then generate a report labeling each control with one of the following states:

OK (green) the measured value is below the threshold
tolerant (yellow) the measured value is above the threshold but within the tolerance range
failed (red), the measured value is above the threshold and the tolerance level.

Figure 1: The goals-and-controls mode, a simplified analyze-and-reflect mode, where you can replace the reflect part with a set of clearly defined traffic lights like controls. For each of the metrics that Sokrates measures, you can specify a control with the threshold and tolerance. And then react when the measurements are out of safe range.

For instance, let’s assume that we have defined a control for duplication, setting the threshold to 5% and the tolerance level to 2%. If the measured duplication level is 4%, Sokrotes will label the control as OK (green). For the measured duplication level of 8%, Sokrates will label the control as failed (red). If the measured duplication is 6%, Sokrates will mark the controls as tolerant (yellow).

Configuring Controls

You configure controls as a part of a Sokrates JSON configuration file, as a part of the goalsAndControls section. You can define multiple goals, each associated with one or more controls.

The following example code fragment is taken from the JUnit5 Sokrates configuration file:

{
  "goalsAndControls": [
    {
      "goal": "Keep the system simple and easy to change",
      "description": "Aim at keeping the system size modest (less than 200,000 LOC is good), duplication low (less than 5% is good), files small (no files longer than 1000 LOC is good), and units simple (no units with more than 25 decision points is good).",
      "controls": [
        {
          "metric": "LINES_OF_CODE_MAIN",
          "description": "Total number of lines of main code",
          "desiredRange": {
            "min": "0",
            "max": "200000",
            "tolerance": "20000"
          }
        },
        {
          "metric": "DUPLICATION_PERCENTAGE",
          "description": "System duplication",
          "desiredRange": {
            "min": "0",
            "max": "5",
            "tolerance": "1"
          }
        },
        {
          "metric": "NUMBER_OF_FILES_1001_PLUS",
          "description": "The number of very large files",
          "desiredRange": {
            "min": "0",
            "max": "0",
            "tolerance": "1"
          }
        },
        {
          "metric": "CONDITIONAL_COMPLEXITY_DISTRIBUTION_26_PLUS_COUNT",
          "description": "Number of very complex units",
          "desiredRange": {
            "min": "0",
            "max": "0",
            "tolerance": "1"
          }
        }
      ]
    }
  ]
}

Figure 2: A fragment of a Sokrates configuration file describing the controls.

Controls Report

Based on the configured values, the Sokrates will generate an HTML report, as well as the text file with the analysis results (by default stored in “_sokrates/reports/data/text/controls.txt”).

The example of a Sokrates controls HTML report you can find here.

Figure 3: A screenshot from a Sokrates controls report. For each control metric, the measured value is compared with a defined range and tolerance.

The equivalent text data file you can find here.

goal: Keep the system simple and easy to change
control metric: LINES_OF_CODE_MAIN
status: OK
desired range: [0 - 200000] ±20000
value: 24088
description: Total number of lines of main code

goal: Keep the system simple and easy to change
control metric: DUPLICATION_PERCENTAGE
status: FAILED
desired range: [0 - 5] ±1
value: 10.077475219323231
description: System duplication

goal: Keep the system simple and easy to change
control metric: NUMBER_OF_FILES_1001_PLUS
status: OK
desired range: [0 - 0] ±1
value: 0
description: The number of very large files

goal: Keep the system simple and easy to change
control metric: CONDITIONAL_COMPLEXITY_DISTRIBUTION_26_PLUS_COUNT
status: OK
desired range: [0 - 0] ±1
value: 0
description: Number of very complex units

Figure 4: A content of a textual data file about controls that Sokrates export.

Metrics Table

2020-05-25T20:12:01+02:00

The metrics table is a list of name-value pairs, a linearized list of all Sokrates aggregated measurements.

Sokrates exports two types of metrics:

the static, project-level metrics. These static metrics have the same name in every Sokrates protect.
derived metrics, where names of a metric are defined based on the names of logical decompositions, components, concerns, or file extensions.

Static Metrics

Static metrics describe the overall system values, such as lines of code in main or test code, or a number of files in the main code. The following table shows some of these metrics:

Figure 1: The screenshot from the Sokrates metrics report, showing some of the static metrics.

Derived Metrics

Derived metrics are dynamic. Their number and names depend on the number and names of components, concerns, and file extensions. The following table shows several of these derived metrics.

Figure 2: The screenshot from the Sokrates metrics report, showing some of the dynamic metrics, where names of metrics depend on component names and file extensions.

These metrics are logical decomposition specific, and Sokrates derives the names of these metrics using these patterns:

NUMBER_OF_FILES_DECOMPOSITION_<LOGICAL_DECOMPOSITION>_<COMPONENT>
LINES_OF_CODE_DECOMPOSITION_<LOGICAL_DECOMPOSITION>_<COMPONENT>
NUMBER_OF_FILES_DECOMPOSITION_<LOGICAL_DECOMPOSITION><COMPONENT>_EXT<EXTENSION>
LINES_OF_CODE_DECOMPOSITION_<LOGICAL_DECOMPOSITION><COMPONENT>_EXT<EXTENSION>

Using Metrics

Sokrates exports all metrics as text, JSON, and HTML files. You can then use these lists for your own processing.

In addition to exporting the list of metrics, Sokrates employs these metrics at two other places:

to define goals and controls. For any metric, you can define the desired range and tolerance and let Sokrates signal if the metric values are in the desired range.
in trend analyses, where you can compare the values of a metric in time