Feature Request: optimized counter?

[whyrusleeping]

For my application i needed a way to track (up to some maximum) the number of likes for each of a number of given post IDs, and I'm already tracking sets of likes on posts by user using roaring bitmaps here, So I implemented a thing thats a fixed size stack of bitmaps that lets me compactly store counts by post ID.

To do the adding, I have a loop that takes in the new bitmap to add in, and then does a series of Xor and AndNot calls to do the 'Add' (see here: https://github.com/whyrusleeping/fancycounter/blob/main/fancycounter.go#L95-L100 )

I'm wondering if theres an optimized way we could do this in a single call (instead of an Xor and an AndNot separately)

Thanks in advance, and thank you so much for this library in the first place!

[lemire]

It could be implemented within the library as part of the BSI component: https://github.com/RoaringBitmap/roaring/tree/master/BitSliceIndexing

I'm wondering if theres an optimized way we could do this in a single call (instead of an Xor and an AndNot separately)

Can you elaborate?

[guymolinari]

What Daniel is saying (I think) is that you can assign a post ID to a positive integer column ID. Then store the count of these in a BSI. There are even functions to increment a counter value. Look at bsi_test.go El mar, 3 de dic de 2024, 6:42 p. m., Daniel Lemire < ***@***.***> escribió:

…

It could be implemented within the library as part of the BSI component: https://github.com/RoaringBitmap/roaring/tree/master/BitSliceIndexing I'm wondering if theres an optimized way we could do this in a single call (instead of an Xor and an AndNot separately) Can you elaborate? — Reply to this email directly, view it on GitHub <#463 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADZZAUQCETMINS24EH4U27L2DZFZFAVCNFSM6AAAAABS66C3CSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMJVHA4DGMZQGA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[whyrusleeping]

I'm not entirely sure I understand what the purpose of a BSI is.

The data structure i need is roughly a map[PostID]int to count likes per post, and a fast way to sum two of these (or add in a []PostID efficiently)

[guymolinari]

A BSI supports more complex data representations that are not possible using a roaring bitmap alone. It supports the sums, GT, LT, GE, LE, and range comparisons of integer values. Think of it as an array of bitmaps where each element of the array represents a digit in a binary number. These numbers can be arbitrarily large.

…

On Tue, Dec 3, 2024 at 8:27 PM Whyrusleeping ***@***.***> wrote: I'm not entirely sure I understand what the purpose of a BSI is. The data structure i need is roughly a map[PostID]int to count likes per post, and a fast way to sum two of these (or add in a []PostID efficiently) — Reply to this email directly, view it on GitHub <#463 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADZZAUX24Q6Z5SC6KMKLKXT2DZSBXAVCNFSM6AAAAABS66C3CSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMJWGAZDSOJXGE> . You are receiving this because you commented.Message ID: ***@***.***>

[lemire]

I'm not entirely sure I understand what the purpose of a BSI is.

I cannot be certain, but it seems that what you have implemented is the same data structure as our BSI.

If that's the case, it might be a good idea to adopt our BSI and contribute to it if needed.