GreptimeDB Roadmap 2023 #669
Description
Overview
GreptimeDB set off in April 2022 and had just open-sourced on Nov. 15th, 2022. Many of our ideas are still on their way, yet to be polished. As we work towards GreptimeDB 2023, it's natural to ask what's next. This roadmap presents goals our team is working on and concepts we have for our shared community.
- To give people a sense of what to expect in GreptimeDB over the next 6 months;
- for those who would like to contribute to GreptimeDB, to help provide "starting points" for how to get involved, and a sense of what kind of projects we are looking for.
We will focus on three main sections:
- Distributed solution at the production level
- Deeply optimize storage and query engine
- Compatible with open protocols
Everyone can easily find all the issues&PRs related to this Roadmap here
Details
Distributed solution at the production level
Features
- Incorporate Quorum-based WAL (as an independent subproject): [Proposal] Distributed write-ahead logging #626
- Analyze Query: support EXPLAIN ANALYZE <QUERY> clause #608
- Design observability system tables: system tables for inner metrics #602
- Enable MySQL/PostgreSQL TLS (suit for new contributors): TLS support for database protocols #515
- Improve fault tolerance using distributed DDL framework Tracking issue for the procedure framework #286
- Integration tests feat: integration test suite #487
Refactor
- Reconstruct gRPC, the data transmission protocol
Deeply optimize storage and query engine
Features
- Add user-defined window functions Implement window functions #103
- Escalate compaction L0 up to L1, & involve TTL Implement Time-To-Live table attribute #601
- Allow multi-region in engine Supports multi-region #625
- Enable object storage, for example, S3 Supports S3 storage in table engine #652
- Continuous aggregating and downsampling Continuous aggregates / downsampling #638
Refactor
- Refact DataFusion & Arrow related components Tracking Issue: migrate Arrow/Parquet to the official implementation. #555
- Incorporate time series index
Performance
- Optimize memtable
- Improve distributed queries
- Add SST's filter & index
- Optimize python coprocessor
Compatible with open protocols
Features
- Enable PromQL (suit for new contributors) Supports PromQL natively #596
- Permit PreparedStatement (suit for new contributors) Prepared statement support in MySQL and PostgreSQL #470
- Be capable of Session (suit for new contributors) session support #600
- Provide SDK of multi-language (suit for new contributors)
How you can help
If anything above draws your attention, don't hesitate to join us on GitHub Discussion, or Join GreptimeDB Community on Slack if you have doubts and would like to talk about it first.
Looking beyond the initiatives that are in progress, there's a lot of room for improvement. Here are some other ideas we'd like to see. For many of these ideas, the main thing they need is someone to own the design! If you might be interested in giving that a try, speak up and chat with the team. We probably will end up being the ones who get you the best.