Skip to main content

Workshop Overview

If you want to build Another Insight tool with another data source rather than GitHub, then you're in the right place. This workshop will tell you it's not too hard to get insights from big data, expecially big data with realtime insert/update.

What you will learn

1. Methodology to setup an insight system

Usually, with enough abstration, there will be 3 steps to setup an insight system like OSS Insight:

  1. Find its Data Source (both historical data and new realtime data)
  2. Load Data to TiDB (or any other HTAP database)
  3. Get Insights with SQL

2. Knowledge about HTAP database

Use HTAP database here is suitable. HTAP stands for Hybrid Transactional and Analytical Processing, it is SUPER EASY to handle both:

  • acting as a primary rdbms to serve requests in high concurrency and insert/update in realtime
  • providing analytical ability to get insights

There is a 10-minute post to explain why we choose tidb to support ossinsight. but for saving time we put a picture below which shows the architecture difference between "before" and "after" we use TiDB:

Simplified architecture after we use TiDB
before and after we use TiDB

That means, we would never need to learn traditional big data skills such as: MySQL+sharding technologies -> industrail etl tools -> olap databases->MySQL(write back), and then setup/manage these infrastructures, but just get a T+1 analysis result finally,


You can load data to MySQL instead of TiDB, but you should will meet with performance issues soon...

Well, for saving time, join the bellow workshops asap, each one will follow the above 3 steps! 🏃🏃🏃

Join Workshop!

We have implemented the mini OSS Insights workshop, and found the posibility of the other 3 workshops, you can try them by yourself with their historical and realtime api (a bit of a challenge, but not too hard :-).

If you want a further talk about OSS Insight, please join our offline workshop, you may get help there:

Apply !