Class: Dis::Storage

Inherits:
Object
  • Object
show all
Defined in:
lib/dis/storage.rb

Overview

Dis Storage

This is the interface for interacting with the storage layers.

All queries are scoped by object type, which will default to the table name of the model. Take care to use your own scope if you interact with the store directly, as models will purge expired content when they change.

Files are stored with a SHA1 digest of the file contents as the key. This ensures data is deduplicated per scope. Hash collisions will be silently ignored.

Layers should be added to Dis::Storage.layers. At least one writeable, non-delayed layer must exist.

Class Method Summary collapse

Class Method Details

.delayed_delete(type, hash) ⇒ Object

Deletes content from all delayed layers.

Dis::Storage.delayed_delete("things", hash)


107
108
109
110
111
 
# File 'lib/dis/storage.rb', line 107

def delayed_delete(type, hash)
  layers.delayed.writeable.each do |layer|
    layer.delete(type, hash)
  end
end

.delayed_store(type, hash) ⇒ Object

Transfers files from immediate layers to all delayed layers.

Dis::Storage.delayed_store("things", hash)


43
44
45
46
47
48
 
# File 'lib/dis/storage.rb', line 43

def delayed_store(type, hash)
  file = get(type, hash)
  layers.delayed.writeable.each do |layer|
    layer.store(type, hash, file)
  end
end

.delete(type, hash) ⇒ Object

Deletes a file from all layers. Kicks off a Dis::Jobs::Delete job if any delayed layers are defined. Returns true if the file existed in any immediate layers, or false if not.

Dis::Storage.delete("things", hash)
# => true
Dis::Storage.delete("things", hash)
# => false


92
93
94
95
96
97
98
99
100
101
102
 
# File 'lib/dis/storage.rb', line 92

def delete(type, hash)
  require_writeable_layers!
  deleted = false
  layers.immediate.writeable.each do |layer|
    deleted = true if layer.delete(type, hash)
  end
  if layers.delayed.writeable.any?
    Dis::Jobs::Delete.enqueue(type, hash)
  end
  deleted
end

.exists?(type, hash) ⇒ Boolean

Returns true if the file exists in any layer.

Dis::Storage.exists?("things", hash) # => true


53
54
55
56
57
58
59
 
# File 'lib/dis/storage.rb', line 53

def exists?(type, hash)
  require_layers!
  layers.each do |layer|
    return true if layer.exists?(type, hash)
  end
  false
end

.get(type, hash) ⇒ Object

Retrieves a file from the store.

stuff = Dis::Storage.get("things", hash)

If any misses are detected, it will try to fetch the file from the first available layer, then store it in all immediate layer.

Returns an instance of Fog::Model.

Raises:


69
70
71
72
73
74
75
76
77
78
79
80
81
 
# File 'lib/dis/storage.rb', line 69

def get(type, hash)
  require_layers!
  miss = false
  layers.each do |layer|
    if result = layer.get(type, hash)
      store_immediately!(type, result) if miss
      return result
    else
      miss = true
    end
  end
  raise Dis::Errors::NotFoundError
end

.layers ⇒ Object

Exposes the layer set, which is an instance of Dis::Layers.


22
23
24
 
# File 'lib/dis/storage.rb', line 22

def layers
  @layers ||= Dis::Layers.new
end

.store(type, file) ⇒ Object

Stores a file and returns a digest. Kicks off a Dis::Jobs::Store job if any delayed layers are defined.

hash = Dis::Storage.store("things", File.open('foo.bin'))
# => "8843d7f92416211de9ebb963ff4ce28125932878"


31
32
33
34
35
36
37
38
 
# File 'lib/dis/storage.rb', line 31

def store(type, file)
  require_writeable_layers!
  hash = store_immediately!(type, file)
  if layers.delayed.writeable.any?
    Dis::Jobs::Store.enqueue(type, hash)
  end
  hash
end