Tech KPIs

💻 Service Delivery & Software Engineering (Execution Focus)

These metrics are essential for measuring the efficiency and quality of your SWE services, aligning with SCRUM velocity and SAFe value streams.

Category	OKR Examples	KPI Examples
Development Velocity & Throughput	O: Dramatically improve the team's capacity to deliver user stories per sprint. KR: Increase the average Sprint Velocity by 10 points over three consecutive sprints.	Sprint Velocity (points completed), Cycle Time (time from commit to deploy), Deployment Frequency, Lead Time for Changes (DORA metric).
Code Quality & Stability	O: Achieve a state of near-zero defects in all production deployments. KR: Reduce the Production Defect Density (bugs per 1000 lines of code) by 40%.	Defect Escape Rate (bugs found in production vs. testing), Mean Time to Recover (MTTR), Test Coverage Percentage, Code Complexity Score (e.g., Cyclomatic Complexity).
Architectural Health	O: Ensure the consultancy's reference architecture is modern and resilient. KR: Successfully migrate 100% of legacy client applications to a containerized microservice architecture.	Technical Debt Index, Service Uptime/Availability, Average Latency/Response Time, Security Vulnerability Count (Severity-weighted).

---

accountability metrics

In Agile project management, accountability metrics are used to measure various aspects of a team's performance and the progress of a project. Here are explanations of some commonly used accountability metrics in Agile:

1. Cycle Time: Cycle time is the duration it takes for a user story or task to move through the entire workflow, from initiation to completion. It measures how quickly the team can deliver work items. By tracking cycle time, teams can identify bottlenecks, improve efficiency, and make more accurate forecasts. Shorter cycle times generally indicate faster delivery and increased productivity.

2. Velocity: Velocity is a measure of a team's productivity and capacity to deliver work within a given time frame, typically measured in story points or units of effort. It represents the average amount of work completed in each iteration or sprint. Velocity is used for estimating how much work can be accomplished in future sprints and for tracking the team's progress over time. It helps with predicting project timelines and managing workload.

3. Predictability: Predictability is the ability to consistently deliver work items within a specified timeframe. It is a measure of how reliable and accurate the team's estimates and commitments are. Predictability metrics assess the variance between planned and actual delivery, evaluating the team's ability to meet deadlines and achieve their commitments. It enables stakeholders to have confidence in the team's ability to deliver on time.

4. Escaped Defects: Escaped defects are bugs or issues that were not discovered during the development and testing process but were identified and reported by users or customers after the software was released. Tracking escaped defects helps assess the quality of the delivered product and the effectiveness of the team's quality assurance processes. It guides improvements in testing, code review, and other quality practices to reduce the number of defects that make it into production.

These accountability metrics provide insights into the team's performance, efficiency, and the overall health of the project. They help identify areas for improvement, enable better decision-making, and facilitate effective planning and forecasting. It's important to note that these metrics should be used in conjunction with other qualitative assessments and should be interpreted in context to gain a comprehensive understanding of the team's progress and capabilities.

Measuring Software Development Productivity

A good developer is able to take a high-level problem, see best way to break it down, and create the correct levels of abstraction, all while keeping the code readable and maintainable for other developers. This also explains why some people are 10x performers, and some people get so frustrated with programming that they give up. Some people have curated, or have a natural talent for, thinking at this extreme level of detail. Some people can intuit things that others will never discover — even if they had all the time in the world. This is the nature of knowledge work.

Anyone who is charged with managing and leading developers is left in a quandary. We want to be fair, but if everything is subjective, how can we be?

Developer Productivity

• Does the developer get a reasonable amount of work done in a given period of time?
• Is the developer's velocity on bug fixing sufficient?

Engagement

• Is the developer dedicated to his/her craft?
• Is the developer committed to delivering software on time?
• Is the developer dedicated to company success?

Attention to Quality

• To what degree does the developer's code work as designed?
• Does the developer thoroughly test code and believe it to be correct before checking it in?
• Do a minimal number of bugs get reported against his/her code?
• Does the developer write unit tests for all new code?
• Does the developer follow the Boy Scout Rule and leave a module cleaner than it was before he or she worked on it?

Code Base Knowledge and Management

• To what degree does the developer understand the code base assigned to him/her?
• Does the developer take responsibility for his/her team's code base, improving it at every opportunity?
• Adherence to coding guidelines and techniques
• Does developer's code routinely meet coding standards?
• Do code reviews reveal a minimum of problems and discrepancies?
• Does the developer use Dependency Injection to ensure decoupled code?

Learning and Skills

• Is the developer constantly learning and improving his/her skills?
• Does the developer show a passion for the craft of software development?

Personal Responsibility

• Does the developer first assume that the error lies within his or her code?
• Does the developer understand that he or she is solely responsible for their code working correctly?
• Does the developer take pride in their code, ensuring it is clean, tested, easy to read, and easy to maintain?

SLOC: source lines of code

1. the number of features developed per month
2. the number of bugs introduced in the code base and the time spent solving those bugs
3. the evolution of the technical debt

50 lines of production code a day

The reality is that only a fraction of a developer's time is actually spent writing production code.

It is generally not considered a valid way to measure productivity. Smaller code is usually better than larger code, so a more productive developer usually produces less code. Productivity takes its biggest hit in debugging; efficient developers spend little time debugging.

its always contextual

The problem with SLOC

The problem with the SLOC metric is that it measures an approximation of the quantity of code written, without taking into account:

• the quality of the code (i.e. what if for every 100 SLOC you have to add another 90 SLOC because of bugs, but that you don't know at the moment your code is delivered?)
• the goals reached with the code (i.e. do the 10K SLOC handle all the expected use cases or user stories? or only a tiny subset?)
• the maintainability of the code (i.e. will you have to add 1% or 50% more code for adjusting the code to expectable evolving requirements?).

Otherwise stated, the production of error prone unmaintainable spaghetti code with lots of copy-pasted parts will be considered as more productive than carefully engineered reusable code.

your team will just game the metrics without producing any meaningful work

So SLOC is definitively not the best way to measure productivity

measuring software development productivity needs to take into account the whole process, including analysing requirements, designing what to code, coding, testing, debugging, and verifying that user expectations are met. As all these activities are very different, the best thing is to measure the only think that matters: working software, i.e. what the software produced means to the user.

How to measure software deliverables ?

Several approaches exist:

• The typical approach in classical software engineering is Function Points (FP). Function points are measured based on the requirements to fulfill (e.g. number of forms, number of fields in each forms, etc...). Productivity is then measured in FP per unit of time and per person. Some companies even have data that telling how many function points a developper can produce per unit of time in a given language for a given domain. The problem with FP is that it requires very detailed requirements upfront and it's time consuming.
• A more modern and pragmatic approach is story points(SP). These are used to evaluate the complexity of the code to be produced, and are routinely used for evaluating velocity of development teams. However, SP is an estimation measure for work performed before all the details are known. It's not a final measure of what actually happened. So some care must be taken when using it as a productivity measure because it could backfire on the estimation process.
• code churn
• Commit frequency
  • 6 commits per week (team activities take up time)

problems with PRS merged with no approver?

Allowing pull requests (PRs) to be merged without an approver can lead to several problems in the software development process. Here are some potential issues:

1. Lack of Code Review: Without an approver, there is no mandatory code review process before merging a PR. Code reviews are essential for maintaining code quality, identifying bugs, and ensuring adherence to best practices. By bypassing code reviews, the merged code may contain errors, security vulnerabilities, or suboptimal implementations.

2. Quality Control: Approvals serve as a quality control mechanism to ensure that the code being merged meets the project's standards. Without an approver, the responsibility for ensuring code quality falls solely on the developer creating the PR. This can result in inconsistent quality standards across the project and increase the likelihood of introducing technical debt.

3. Lack of Knowledge Sharing: Code reviews provide an opportunity for knowledge sharing and learning within the development team. By having multiple team members review and provide feedback on PRs, knowledge about different parts of the codebase is disseminated. Without an approver, this valuable learning opportunity is missed, potentially leading to knowledge silos and reduced collaboration.

4. Missed Collaboration and Feedback: The review process allows for collaboration and discussion between developers, leading to better solutions, alternative approaches, and improved code. Without an approver, this collaboration may be limited, and valuable insights and feedback from experienced team members may be missed.

5. Decreased Accountability: Having an approver in the PR workflow adds an extra layer of accountability. It ensures that someone other than the author of the code has reviewed and approved the changes. Without an approver, the responsibility for the quality and correctness of the code rests solely on the developer creating the PR, potentially leading to a lack of accountability and a higher risk of errors going unnoticed.

6. Reduced Consistency and Standards: Approvals help enforce coding standards and consistency across the codebase. They ensure that the code follows established guidelines and best practices. Without an approver, there is a greater chance of inconsistencies, divergent coding styles, and deviations from established conventions.

It's important to note that the absence of an approver doesn't necessarily mean that the code will be of poor quality or problematic. However, having a review and approval process in place contributes to a more robust and reliable development process by leveraging the expertise and collaboration of the development team.

PRs open for more than 24 hrs?

Keeping pull requests (PRs) open for an extended period, such as more than 24 hours, can lead to several problems in the software development process. Here are some potential issues:

1. Delays in Feedback and Collaboration: PRs are meant to facilitate collaboration and feedback among team members. By keeping PRs open for an extended period, it delays the feedback loop and hampers effective collaboration. Developers may have to wait longer for reviews, causing delays in resolving issues, making necessary changes, and progressing with the development work.

2. Increased Merge Conflicts: Long-lived PRs increase the likelihood of merge conflicts. As other changes are made to the codebase during the PR's lifetime, it becomes more challenging to merge the changes cleanly. Resolving merge conflicts can be time-consuming and may introduce errors or inconsistencies if not handled carefully.

3. Reduced Agility and Iteration Speed: Agile methodologies emphasize quick iterations and continuous delivery. If PRs remain open for an extended period, it hinders the ability to deliver smaller, incremental changes frequently. This can impact the agility of the development process and delay the release of new features or bug fixes.

4. Context Switching and Loss of Focus: When PRs are left open for an extended duration, developers may lose context and focus on the changes they made. They may need to spend additional time re-familiarizing themselves with the code and the reasons behind their changes. This can slow down the development process and potentially introduce errors due to the loss of concentration.

5. Increased Risk of Code Decay: The longer a PR remains open, the greater the chance that the codebase it is based on becomes outdated. Other changes and updates may have been made to the codebase, making the PR less compatible or causing conflicts. This can result in rework, additional testing, and potential regressions when merging the PR.

6. Reduced Accountability and Ownership: Long-lived PRs can lead to a lack of accountability and ownership. Developers may be less motivated to address issues promptly or take responsibility for their changes if the PR remains open for an extended period. This can impact the overall code quality and hinder the timely resolution of problems.

While there may be valid reasons for PRs to remain open for more than 24 hours in certain situations, it is generally beneficial to keep PRs as short-lived as possible. Regularly reviewing and addressing PRs helps maintain a smooth and efficient development process, promotes collaboration, and reduces the risk of issues arising from delays and code divergence.