An alternative tuple-storage engine for Casbin Mesh / Casbin — GSOC 2022 Proposal

Currently, Casbin uses golang built-in map structure to maintain policies in the main memory and persist the policies via adapter abstraction. If policies data grows, however, the growing cost of main memory resources and bad performance make the memory management strategy not tolerable anymore. We need to find a better way to manage the casbin in-memory data when data grows. From my point of view, our main goal is to reduce the cost of memory as well as keep good performance handling policies read and write requests. In order to achieve those key goals, we can introduce an experimental tuple storage to get charge of storing those policies, turning the policies management strategy from memory-oriented to disk-oriented. We can even make a better abstraction of the storage layer so that we can use different engines (row, column) for the different workloads. In general, we can take the following parts into consideration to achieve our goals. 1. API for upper layer. 2. workload optimizer. 3. Buffer Pool management. 4. Indexing. 5. Data Storage Structures. 6. Transaction if necessary. And in the end, the deliverables will be : A Casbin built-in embedded disk-oriented tuple storage engine. The Engine should contain ： 1. Carefully designed API for upper Casbin internal module. 2. Storage management. Include file organization and page layout. 3. Buffer pool management. 4. Index management. 5. An workload optimizer for the upper layer. 6. Transaction part if necessary.