Track ZooKeeper property creation and modification times#6462
Track ZooKeeper property creation and modification times#6462DomGarguilo wants to merge 1 commit into
Conversation
|
Before merging, I want to discuss the trade off of storing this extra metadata in zookeeper. While it can be helpful to see the create and edit timestamps for individual props, we have to assess whether its worth the increased amount of data written in zookeeper per prop. The props blob is gzip compressed so things should compress reasonably well but we are still adding two extra timestamps per key. I do not have a great idea how much this will affect things in a real-world deployment so if anyone has an insights/opinions on things here that would be helpful. |
|
The way I think about the utility of this is analogous to a config file. Each of these blobs of config have a version (modified time) when they are modified already, similar to a config file's modification time on a filesystem. We don't generally expect to be able to open a config file on a filesystem and check the timestamp of when each line in the file is modified, and that's fine, because we don't really need that. If we do, we depend on an external service to monitor changes as patch files or similar (like svn or git). Even if the extra storage isn't that much in ZK, there's also a question of increased code complexity vs. utility. I think the utility of having the granular versioning baked in is low relative to the increased code quantity/complexity. It is probably simpler, and satisfies user interest better, to just log configuration changes on the server-side, when an RPC request comes in to change the config. Whether it's an API request to remove a property, set a property, or modify the properties, we can easily log the request on the server-side in a very simple manner, without the complexity of tracking the metadata for when and what was changed. Users can collect and analyze those logs if they want to maintain a history of changes. That could also be done very easily in 2.1 also. That said, I need to spend a little more time looking into this implementation. I was thinking about a couple of edge cases, and need to look more carefully to see how this PR handles those. |
Fixes #6254
createdandmodifiedmetadata toVersionedPropertieszk info-viewer --print-propsHere is some example output from running on a test table "ts_meta_test" with one custom prop set
accumulo zk info-viewer --print-props -t ts_meta_test:Here is some pasted lines from some more testing where you can see updating of the
createdandmodifiedtimes.in the shell I ran this:
and in between each step I printed the output:
You can see the initial set creates matching timestamps:
You can see adding a second property gives it its own timestamps:
You can see changing owner preserves created and updates modified:
You can see setting owner=domG again does not update modified:
You can see deleting table.custom.property removes it from the output: