A New Approach to Metrics

Honeycomb

And it includes metrics. Time comparisons let you compare query results over two time ranges. With time comparisons, you can quickly compare traffic patterns to that same time period day over day, or week over week. Does that mean we’re no longer meh on metrics?

Getting Metrics Into Honeycomb

Honeycomb

Rather than relying on the rigid pre-aggregated measures provided by metrics, Honeycomb instead favors using events because they provide a much greater degree of flexibility. But that doesn’t mean metrics don’t belong in Honeycomb. The diminishing value of metrics.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Research Results: Key software architecture metrics

Apiumhub

If you remember my article about Software Architecture Quality Attributes , you know that we have been conducting a survey to find out key software architecture metrics that leading companies and software architects use. Results: key software architecture metrics.

Agile Metrics — The Good, the Bad, and the Ugly

Scrum.org

TL; DR: Agile Metrics. Suitable agile metrics reflect either a team’s progress in becoming agile or your organization’s progress in becoming a learning organization. At the team level, qualitative agile metrics often work better than quantitative metrics.

Performance comparison of video coding standards: an adaptive streaming perspective

Netflix TechBlog

Chef’s Table Factors in codec comparisons Many articles have been published comparing the performance of video codecs. We discuss below several factors that impact the assessment of video codecs: Encoder implementation Encoder settings Methodology Content Metrics Where applicable, we make the distinction between the traditional comparison approach and our approach for adaptive streaming. It can use any metric to guide its optimization process.

AWS vs Azure vs Google Free Tier Comparison

ParkMyCloud

Whether you’re new to public cloud altogether or already use one provider and are interested in trying another, you may be interested in a comparison of the AWS vs Azure vs Google free tier. The post AWS vs Azure vs Google Free Tier Comparison appeared first on ParkMyCloud.

Project Success: Time To Get Your Metrics Right

Mentormate

Descriptive criteria include metrics and/or targets to reach at delivery. Reliable measurement of each of the chosen success criteria can be achieved if all stakeholders are aligned with the defined metrics, the frequency of measurement, and roles associated with it (who will perform, track and enforce the measurement process). To achieve efficiency, the whole team should be involved in defining the measurement map (metrics – frequency – roles).

EV/EBITDA ratio

Agile Lucero

The EV/EBITDA ratio is a comparison of enterprise value and earnings before interest, taxes, depreciation and amortization. This is a very commonly used metric for estimating the business valuations. Financial Metrics

The Acid Ratio

Agile Lucero

This metric is more useful in certain situations than the current ratio, also known as the working capital ratio , since it ignores assets such as inventory, which may be difficult to quickly liquidate. In such cases, other metrics should be considered, such as inventory turnover.

The Big Picture of Business – Quality is Important for Business: Real Quality vs. Arbitrary Metrics

Strategy Driven

They use the term ‘metrics’ out of context. Their metrics are arbitrary, and they jerk the chains of sellers with figures that are unsubstantiated. Metrics are easily skewed and do not reflect the overall customer satisfaction. A criticism of performance metrics is that when the value of information is computed using mathematical methods, it shows that even performance metrics professionals choose measures that have little value.

Data-Informed Retrospectives

Scrum.org

An additional metric might be the amount of anticipated work at the beginning of the Sprint vs. done work delivered at the end of the Sprint. Another good source for actionable agile metrics is the State of DevOps Report.). The Usefulness of Team Metrics.

Data 174

Am I Smart or Just Lucky? Understanding Your Process Integrity Risk with Tenable Lumin

Tenable

Business system risk and process integrity risk are two essential metrics for a mature risk-based vulnerability management practice. Risk-based vulnerability management requires metrics addressing two types of risks: Business system risk. Assessment maturity metrics.

Key Challenges with Quasi Experiments at Netflix

The Netflix TechBlog

In summary, simple randomization and mean comparison we typically utilize in A/B testing with millions of members may not work well for quasi experiments. getting closer to apples to apples comparison). For a new show, we start observing metrics only when the show launches.

Myth: Velocity is Productivity

Scrum.org

The few metrics it produces make little sense outside of the context of team planning. There are no credible means of translating it into a normalized figure that can be used for meaningful comparison. . So, Am I saying metrics are bad?

SCRUM 157

5 Factors to Consider When Choosing a Stream Processing Engine

Cloudera

but have you really examined the stream processing engines out there in a side-by-side comparison to make sure? Are you using the right stream processing engine for the job at hand? You might think you are—and you very well might be!—but

Detecting Performance Anomalies in External Firmware Deployments

Netflix TechBlog

While we have tools for dealing with the first scenario (for example, automated canary analysis using Kayenta ), the second type previously was only detected when the update had reached a sufficient volume of devices to shift core performance metrics. Putting the Pieces Together Consider the case of a metric which follows the grey distribution (with a mean value of ~ 4,570) shown in Figure 2.

Scrum vs. Kanban: Which Agile Framework Is Best?

Gorilla Logic

Scrum and Kanban are each a framework that teams can use to define roles and responsibilities, structure deployment, release cadence, and company metrics. Metrics in Scrum in comparison to Kanban.

SCRUM 75

70 Scrum Master Theses

Scrum.org

On the other hand, the Scrum Master Theses also cover, for example, the relationship with the Product Owner, they deal with agile metrics, and how to kick-off an agile transition, thus moving beyond the original framework of the Scrum Guide. Agile Metrics.

SCRUM 141

Running Code and Failing Models

DataRobot

For comparison, a random forest model achieves 2.38 and Google benchmarks for this model use the accuracy metric. Although it’s easy to get running code with the accuracy metric, it’s not good data science practice for this problem. Machine learning is a glass cannon.

Octoverse spotlight: An analysis of developer productivity, work cadence, and collaboration in the early days of COVID-19

Github

A complex construct such as productivity can’t be captured with simple or single metrics like lines of code or issues closed. This means our year-over-year comparisons are reliable. This makes year-over-year comparisons unreliable for people who were not previously working from home.

New benchmarks for approximate nearest neighbors

Erik Bernhardsson

What’s bothered me about the research is that there’s a thousand papers about how to do this, but very little empirical comparison. glove-100-angular: sift-128-euclidean: fashion-mnist-784-euclidean: gist-960-euclidean: In almost all the datasets the top 5 are in the following order: HNSW (hierarchical navigable small world) from NMSLIB (non metric search library) knocks it out of the park. UPDATE(2018-06-17): There are is a later blog post with newer benchmarks !

In defense of false positives (why you can't fail with A/B tests)

Erik Bernhardsson

Many years ago, I used to think that A/B tests were foolproof and all you need to do is compare the metrics for the two groups. What to think about when running many A/B tests , why you shouldn’t check your metrics every day , why you shouldn’t optimize for a local maximum, and so on. Even if you don’t care about confidence intervals, multiple comparison corrections, or if you are basically too impatient to wait, you probably are still doing the right thing.

In defense of false positives (why you can't fail with A/B tests)

Erik Bernhardsson

Many years ago, I used to think that A/B tests were foolproof and all you need to do is compare the metrics for the two groups. What to think about when running many A/B tests , why you shouldn’t check your metrics every day , why you shouldn’t optimize for a local maximum, and so on. Even if you don’t care about confidence intervals, multiple comparison corrections, or if you are basically too impatient to wait, you probably are still doing the right thing.

Observability in Azure Using Built-in and Third-Party Tools

Mentormate

As you can see, Azure Monitor uses two fundamental types of data: metrics and logs. As a bold comparison to the DIKW pyramid , Azure Monitor brings users to knowledge.

Azure 69

Benchmarking AWS Lambda runtimes in 2019 (part I)

The Agile Monkey

Hence, we decided to follow previous comparison approaches and perform load testing using Serverless Artillery based on Artillery, but taking advantage of the “infinite scaling” of AWS Lambda to simulate concurrent requests. Serverless Artillery Script AWS CloudWatch helped us collect and plot the metrics for our benchmark. Despite the limitations of the default metrics available, custom widgets allowed us to perform more complex queries based on logs.

How Reliable Is Cable Internet? Here’s How Our Networks Are Performing

CableLabs

Zoom, Microsoft Teams) do require more data to be transmitted from the home (upstream) in comparison with video streaming services due to two-way audio and video functionality. Additional metrics, trends and observations on cable internet usage can be found on NCTA’s COVID-19 Dashboard.

Deciphering Goals

Scrum.org

This blog aims to shed some light on the underlying forces at play and how good goal setting with well-chosen metrics can help. Table 2 Goal instances comparison. Looking at the comparison, FAST goals seem like the better choice when creating high level goals.

How to Make Agile Work in Fast-Growing Startups

Scrum.org

Equally futile by comparison to the lack of C-level support is a bottom-up approach by hacking the existing culture. How-to #7: Metrics in Agile Startups. The general purpose of agile metrics is to understand the current situation better and gain insight into change over time.

Agile 142

Quantifying the value of multi-cloud deployment strategies with CDP Public Cloud

Cloudera

CDP Public Cloud costs are excluded in this comparison because hourly pricing for CDP Public Cloud credits is the same across CSPs for similar instance types (similar by means of instance class and compute capacity in vCPUs).

Cloud 89

Reimagining Experimentation Analysis at Netflix

The Netflix TechBlog

ABlaze: The standard view of analyses in the XP UI Suppose you’re running a new video encoding test and theorize that the two new encodes should reduce play delay, a metric describing how long it takes for a video to play after you press the start button. They can then choose from a library of statistics and visualizations or contribute their own to get a deeper understanding of the metrics. Getting Data with the Metrics Repo 2.

Oracle to Amazon RDS Migration Best Practices

Datavail

You need to gather a variety of data on your current Oracle performance under different conditions to make a fair comparison to Amazon RDS. Keep an eye on your high and low usage numbers, performance bottlenecks, storage requirements, and other metrics. An Oracle to Amazon RDS migration is an extensive process that requires significant resources.

How To Hire The Right Software Developers – 7 Tactics That Actually Works

Xicom

Hiring developers from IT agencies offer behemoth benefits in comparison to freelancers. Let me get you through the hiring benefits of the right software development company in comparison to freelancers.

The Intersection of Climate and Capital Markets

Cloudera

Financial services firms will need to understand their trading book and related capital requirements value against a new set of data that includes climate progress metrics. Large quantities of historical data for comparison and trend analysis. Happy Earth Day!

Interpreting predictive models with Skater: Unboxing model opacity

O'Reilly Media - Data

Further evaluation and validation of models could be enabled using algorithms for model comparison for classification and regression using model specific scorers—AUC-ROC ( area under the receiver operating curve ) and MAE ( mean absolute error ). model comparison and performance evaluation. Analysts and data scientists can possibly use model comparison and evaluation methods to assess the accuracy of the models.

Reclaiming the stories that algorithms tell

O'Reilly Media - Ideas

So the state calculates and publishes a “Risk Adjusted Mortality Ratio”—a comparison between the actual number of observed deaths and the number that would be statistically expected, on average, for patients medically similar to those each doctor actually operated on.

Data Compression for Large-Scale Streaming Experimentation

The Netflix TechBlog

QoE is measured with a compilation of metrics that describe everything about the user’s experience from the time they press play until the time they finish watching. This is a hard problem because we must determine if and how the QoE metric distributions differ between experiences.

Operational Database Performance Improvements in CDP Private Cloud 7 vs CDH5

Cloudera

For this performance comparison, we measured HBase2 available in CDP Private Cloud 7 to Hbase1 available in CDH 5 using YCSB workloads. An application example would be a metrics store.

Data Compression for Large-Scale Streaming Experimentation

The Netflix TechBlog

QoE is measured with a compilation of metrics that describe everything about the user’s experience from the time they press play until the time they finish watching. This is a hard problem because we must determine if and how the QoE metric distributions differ between experiences.

Outreach Engages Their Production Code with Honeycomb

Honeycomb

Wavefront for Metrics. Millions of Logs and Limited View Metrics. The team considered a metrics-based approach for discovering production issues. The Outreach team eventually settled on Wavefront for general metrics.

Earned Value Analysis for Justification of Continuous Value

SCRUMstudy

The result obtained is a simple set of metrics that indicate performance issues and allow timely adjustments. Earned Value Analysis also improves the definition of project scope and offers meaningful metrics that can be communicated to stakeholders. This analysis or comparison helps in estimating future performance of the project. For implementing EVA, some basic metrics such as valuation of planned work (called planned value or PV) need to be measured.