All examples will be done with devcam, cause I didn't want to screw with my real instance. Devcam is just a wrapper around camlistored that creates a temporary blobstore and index so you can experiment and stuff.
➜ camlistore.org git:(master) devcam put file ~/Pictures/mako-screens/Screenshot_2012-12-27-16-00-50.png
sha1-8d87e15e54b9da2cf2b9e65e483107e03b8dfbc7
The hash you get back is the metadata for the file, we can pull it like so.
➜ camlistore.org git:(master) devcam tool describe sha1-8d87e15e54b9da2cf2b9e65e483107e03b8dfbc7
{
"meta": {
"sha1-8d87e15e54b9da2cf2b9e65e483107e03b8dfbc7": {
"blobRef": "sha1-8d87e15e54b9da2cf2b9e65e483107e03b8dfbc7",
"camliType": "file",
"size": 380,
"file": {
"fileName": "Screenshot_2012-12-27-16-00-50.png",
"size": 62675,
"mimeType": "image/png",
"time": "2014-03-14T04:09:46.825709033Z",
"wholeRef": "sha1-cc9fb21e63446b5e6056942b520898c9afa065d5"
},
"image": {
"width": 768,
"height": 1280
}
}
}
}
camtool describe just asks the indexer what it knows about the ref. To view the raw blob use camget.
➜ camlistore.org git:(master) devcam get sha1-8d87e15e54b9da2cf2b9e65e483107e03b8dfbc7
{"camliVersion": 1,
"camliType": "file",
"fileName": "Screenshot_2012-12-27-16-00-50.png",
"parts": [
{
"blobRef": "sha1-cc9fb21e63446b5e6056942b520898c9afa065d5",
"size": 62675
}
],
"unixGroup": "users",
"unixGroupId": 100,
"unixMtime": "2014-03-14T04:09:46.825709033Z",
"unixOwner": "drew",
"unixOwnerId": 1000,
"unixPermission": "0644"
}
to retrieve the actual file use -o
➜ camlistore.org git:(master) devcam get -- -o . sha1-8d87e15e54b9da2cf2b9e65e483107e03b8dfbc7
2015/04/01 12:03:51 Writing sha1-8d87e15e54b9da2cf2b9e65e483107e03b8dfbc7 to Screenshot_2012-12-27-16-00-50.png ...
-o just fetches the meta then loops through and concats all the parts.
also note that the file doesn't show in the webgui this is because it isn't associated with a permanode. I'll get to that later. But you can still find it if you know exactly what your looking for (im not sure about future behavior here, camlistore doesn't have gc yet but it will and my understanding is that blobs not 'claimed' can be gc'd)
➜ camlistore.org git:(master) ✗ devcam tool search --rawquery '{"constraint":{"file":{"fileName":{"equals":"Screenshot_2012-12-27-16-00-50.png"}}}}'
{
"blobs": [
{
"blob": "sha1-8d87e15e54b9da2cf2b9e65e483107e03b8dfbc7"
}
],
"description": null
}
--rawquery is pretty flexible. you can checkout all the possible constraints at https://camlistore.googlesource.com/camlistore/+/master/pkg/search/query.go
Uploading Files with Filenodes
My screenshot is still unclaimed, we can do this manually but i'd rather not (more on this below). So I'll use camput do do it for me.
➜ camlistore.org git:(master) ✗ devcam put file --filenodes --tag="Screen Shot","mako","Nexus 4" ~/Pictures/mako-screens/Screenshot_2014-02-28-11-38-33.png
sha1-65095c3dc3c69d05be69eac0a6c705e9ba8e77c6
sha1-1d6e7cf65556f235becd8660b2d4280454b38e48
sha1-b281f844a329cc4b874e4855754091185fd69b40
sha1-5dabe73fdcb2ef72306bb82bdc285472cfd95142
sha1-42911aa3019889b90a88e9d5bd95652c19beae2a
sha1-05135e0472e025c3ff4937589d199e3ed1ac99db
Notice we get back 6 hashes. The first is usually the created permanode
➜ camlistore.org git:(master) ✗ devcam tool describe sha1-65095c3dc3c69d05be69eac0a6c705e9ba8e77c6{ "meta": { "sha1-65095c3dc3c69d05be69eac0a6c705e9ba8e77c6": { "blobRef": "sha1-65095c3dc3c69d05be69eac0a6c705e9ba8e77c6", "camliType": "permanode", "size": 576, "permanode": { "attr": { "camliContent": [ "sha1-05135e0472e025c3ff4937589d199e3ed1ac99db" ], "tag": [ "Screen Shot", "mako", "Nexus 4" ] }, "modtime": "2014-03-14T04:09:57.255844581Z" } } }}
The next four are mutation claims, one assigning the file as the camliContent. The other three add the tags. The permanode and claims are all just blobs and immutable, its the indexer that puts everything together to make the describe output above. You can mutate the permanode by adding any arbitrary attribute you want to it, but there are several reserved attributes namely; tag, title, and anything starting with camli. All the currently used attrs are listed in the link above.
Permanodes
Now lets pretend like i uploaded several other screenshots using --filenodes and i want to group them into a collection.
Since i tagged them i can easily locate their permanodes
➜ camlistore.org git:(master) ✗ devcam tool search 'tag:mako'{ "blobs": [ { "blob": "sha1-7a6d438affd32d701e153fc2e2b1dfd48cb7cd25" }, { "blob": "sha1-eb1e40d5aa8f758439ca1cc62e9ef35fa2c1a50a" }, { "blob": "sha1-093e4aa28f33b822b21a75614311d6a1a3b43aeb" }, { "blob": "sha1-65095c3dc3c69d05be69eac0a6c705e9ba8e77c6" } ], "description": null}
Now create the permanode under which to group them
➜ camlistore.org git:(master) ✗ devcam put permanode --title="Mako Screens"sha1-cf968b27d78f4b12438b3fb8d1fe4b26b73e269bsha1-ea94b63a4c683a94bc26340c23a58e31fabd0860
And start adding them
➜ camlistore.org git:(master) ✗ devcam put attr --add sha1-cf968b27d78f4b12438b3fb8d1fe4b26b73e269b camliMember sha1-7a6d438affd32d701e153fc2e2b1dfd48cb7cd25sha1-f8679b4937703bdaac03c890c282ee4ec1783f23
This says add the sha1-f867 permanode to the camliMember attribute in the sha1-cf96 (Mako Screens) permanode.
Then pretend i added all the remaining screens to the "Mako Screens" permanode
➜ camlistore.org git:(master) ✗ devcam tool describe sha1-cf968b27d78f4b12438b3fb8d1fe4b26b73e269b{ "meta": { "sha1-cf968b27d78f4b12438b3fb8d1fe4b26b73e269b": { "blobRef": "sha1-cf968b27d78f4b12438b3fb8d1fe4b26b73e269b", "camliType": "permanode", "size": 562, "permanode": { "attr": { "camliMember": [ "sha1-7a6d438affd32d701e153fc2e2b1dfd48cb7cd25", "sha1-eb1e40d5aa8f758439ca1cc62e9ef35fa2c1a50a", "sha1-093e4aa28f33b822b21a75614311d6a1a3b43aeb", "sha1-65095c3dc3c69d05be69eac0a6c705e9ba8e77c6" ], "title": [ "Mako Screens" ] }, "modtime": "2015-04-01T18:14:53.742708604Z" } } }}
And witness that there is a new "Mako Screens" folder in the webui, containing the four screens
Permanode summary
What I've gleened so far is that there are (at least) two kinds of permanodes.
The first, filenode, is a permanode with a "camliContent" attribute that points either to a file or a set (i haven't discussed sets and won't but it essentially a directory). The "camliContent" attribute cannot be another permanode. I should also point out that althought the "camliContent" attribute is an array it only wants one value.
The second, I'll call it a collection, is a permanode with a "camliMember" attribute, all hashes in the "camliMember" array must be other permanodes.
I haven't confirmed this but my reasoning is that the "camliContent" and "camliMember" are mutually exclusive.
Permanode summary
What I've gleened so far is that there are (at least) two kinds of permanodes.
The first, filenode, is a permanode with a "camliContent" attribute that points either to a file or a set (i haven't discussed sets and won't but it essentially a directory). The "camliContent" attribute cannot be another permanode. I should also point out that althought the "camliContent" attribute is an array it only wants one value.
The second, I'll call it a collection, is a permanode with a "camliMember" attribute, all hashes in the "camliMember" array must be other permanodes.
I haven't confirmed this but my reasoning is that the "camliContent" and "camliMember" are mutually exclusive.
Deleting
Now lets delete something. Delete is a pretty strong word here. Because nothing in camlistore is ever deleted. (...yet we'll see when gc is implemented). The best we can do is 'hide' a permanode by issuing a delete mutation.
➜ camlistore.org git:(master) ✗ devcam put delete sha1-cf968b27d78f4b12438b3fb8d1fe4b26b73e269bsha1-775ee5a2bedbf98e21398cce277f6aa8bd95ef3f
And witness it removed from the webui.
This is what our delete claim (mutation) looks like (using camget since describe doesn't tell us what type of claim it is)
➜ camlistore.org git:(master) ✗ devcam get sha1-775ee5a2bedbf98e21398cce277f6aa8bd95ef3f{"camliVersion": 1, "camliSigner": "", "camliType": "claim", "claimDate": "2015-04-01T18:28:06.028725401Z", "claimType": "delete", "target": "sha1-cf968b27d78f4b12438b3fb8d1fe4b26b73e269b","camliSig":""}
But wait... shit. that's not the right one! I meant to delete another permanode. Well thats not a problem. All we have to do is delete the delete mutation. (Its hash was output above)
➜ camlistore.org git:(master) ✗ devcam put delete sha1-775ee5a2bedbf98e21398cce277f6aa8bd95ef3fsha1-f226e2400266ab6240a2825f8ecb8df7381f523b
And Mako Screens is back in the webui. Again notice, it wasn't actually deleted, we just mutated the delete claim. Essentially telling the indexer to ignore it.
Additional note about claims, claim search isn't implemented yet. But you can get a list of claims assossiated with a permanode by pinging the api. (Theres probably another way but I haven't found it yet)
➜ camlistore.org git:(master) ✗ curl localhost:3179/my-search/camli/search/claims\?permanode=sha1-cf968b27d78f4b12438b3fb8d1fe4b26b73e269b
More about claims:
A delete claim can only be issues on a permanode or another claim, it won't work on files (immutable remeber?)
I intermingled the words claim and mutation a bit. In this context they mean the same thing, a mutation is anything with the camliType "claim". A mutation essentially changes a blobref. I'm pretty sure you can mutate any blob but I'm not sure how useful it is. I've only mutated permanodes so far. The most useful mutations I've used are: adding/removing tags, changing title, changing camliMember, changing camliContent, deleting, undeleting, also removing/adding arbitrary attributes to the permanode. Though I'm sure there are other kinds of mutations I haven't used.
When mutating permanode attributes be aware that camlistore has several reserved attributes, namely; tag, title, and anything starting with camli. Further doc at https://camlistore.googlesource.com/camlistore/+/master/doc/schema/claims/attributes.txt
Summary:
At present camlistore is very image oriented which is why my examples used images. It also has the ability to parse mp3s (if you have good id3 tags). But wasn't able to pull meta from videos I added.
Camlistore is all about archival. Everything you add and every change made (mutation) is an immutable blob that's there for life!
The current release is 0.8 but thats really old, building from master is a better option and 0.9 shouldn't be too far away. And it should have importers (there's some code in master but afaik they aren't functional, or at least I didn't see a way to configure them but I also didn't look into it that much).
Sorry if this post seems clumsily put together, camlistore is emmensly complex and trying to write a coherent "intro to camlistore" turned out to be difficult as hell. More posts to come as I did deeper.
Camlistore is all about archival. Everything you add and every change made (mutation) is an immutable blob that's there for life!
The current release is 0.8 but thats really old, building from master is a better option and 0.9 shouldn't be too far away. And it should have importers (there's some code in master but afaik they aren't functional, or at least I didn't see a way to configure them but I also didn't look into it that much).
Sorry if this post seems clumsily put together, camlistore is emmensly complex and trying to write a coherent "intro to camlistore" turned out to be difficult as hell. More posts to come as I did deeper.
No comments:
Post a Comment