System Design Look Back - A Browser Monitor System

Days ago, I interviewed for a position in Microsoft and the interviewer asked how to design a browser monitor system that makes sense for Bing.com. Basically, it is an open-ended system design question that can be delved into with lots of aspects.

In the domain of browser monitor systems, there are tools that are very popular in the industry and provide end-to-end services like Sentry.io (opens in a new tab), LogRocket (opens in a new tab), Datadog (opens in a new tab), and MicroSoft Clarity (opens in a new tab), etc.

Some critical product components include:

1. client SDK: event; performance; error; replay; route;
2. data format: JSON; ProtoBuf; MessagePack; Thrift;
3. data process and storage: Kafka; ClickHouse; PostgreSQL
4. data management: session replay; product heatmap; error tracking; performance waterfall; customize SQL Graph
5. alert and monitor: data threshold; third-party integration; cron job

The monitor system is complex and valuable for products especially in Big companies since this type of data represents your product and not every company wants to share its secret with Sentry.io. Based on this, instead of using a public service company like Bytedance rather hire a group of engineers at the cost of billions of dollars to fork and develop the open-source version of Sentry.io.

Browser Monitor SDK

Sentry.io is the leader of this domain, the source code (opens in a new tab) of the SDK is necessary to be researched.

There are 2 core components in Sentry Javascript browser SDK:

1. Tracing (opens in a new tab)

• performance metrics: LCP, FID, CLS, FP, FCP, RTT, longtask, etc.
• network: fetch, XHR, resource
• event: click, scroll, hover, navigation, visibility, exit, etc.
• error: JS error, network error, crash.
• data assemble: aggregation, time threshold, config

2. Intrgration (opens in a new tab)

• session replay
• custom feedback
• log capture
• etc

The SDK codebase is easy enough that you can copy the majority of the codebase to rebuild a new monitor system. The key problem is from the backend on how to integrate the data into your business model and make it customized. Maybe you have a complex e-commerce system, medical system, CMS system, etc. The back monitor system is completely isolated from your business backend, and you might need to hire more engineers to handle your requirements like Bytedance.

End

The value of a customized monitor system is critical for a serious business. For a start-up, shipping a new feature ASAP is the first consideration, and using the industry standard is the optimal choice.

Building a such system is fun for engineers since there are so many domains that can be delved into and improve efficiency and save resources aside from growing your actual business.