djblets.cache.backend¶
Utility functions for working with memory caching backends.
These functions are designed to integrate with a cache backend using Django’s cache framework. They handle creating caching keys unique to the install and caching more complex data efficiently (such as the results of iterators and large data normally too big for the cache).
- cache_memoize_iter(key, items_or_callable, expiration=2592000, force_overwrite=False, compress_large_data=True, use_encryption=None, encryption_key=None)[source]¶
Memoize an iterable list of items inside the configured cache.
If the provided list of items is a function, the function must return a an iterable object, such as a list or a generator.
If a generator is provided, directly or through a function, then each item will be immediately yielded to the caller as they’re retrieved, and the cached entries will be built up as the items are processed.
The data is assumed to be big enough that it must be pickled, optionally compressed, and stored as chunks in the cache.
Data can be encrypted using AES encryption, for safe storage of potentially sensitive information or state. This adds to the processing time slightly, but can be important for cache keys containing sensitive information or that impact access control, particularly in the event that the cache is compromised or shared between services.
The result from this function is always a generator. Note that it’s important that the generator be allowed to continue until completion, or the data won’t be retrievable from the cache.
Changed in version 3.0: Added support for encrypting keys and data through the
use_encryptionandencryption_keyarguments.- Parameters:
key (
unicode) – The key to use in the cache.items_or_callable (
listorcallable) – A list of items or callable returning a list of items to cache, if the key is not already found in cache.expiration (
int) – The expiration time for the key, in seconds.force_overwrite (
bool) – IfTrue, the value will always be computed and stored regardless of whether it exists in the cache already.compress_large_data (
bool) – IfTrue, the data will be zlib-compressed.use_encryption (
bool, optional) –Whether to use encryption when storing or reading data.
If reading data, and if the data cannot be decrypted with the given key, then the data will be considered to have fallen out of cache.
This defaults to
False, but can be forced on for all cached data by settingsettings.DJBLETS_CACHE_FORCE_ENCRYPTION=True.New in version 3.0.
encryption_key (
bytesorstr, optional) –An explicit AES encryption key to use when passing
use_encryption=True.This defaults the value in
settings.DJBLETS_CACHE_DEFAULT_ENCRYPTION_KEY, if set, or to the default AES encryption key for the server as provided bydjblets.secrets.crypto.get_default_aes_encryption_key().New in version 3.0.
- Yields:
object– The list of items from the cache or fromitems_or_callableif uncached.
- cache_memoize(key, lookup_callable, expiration=2592000, force_overwrite=False, large_data=False, compress_large_data=True, use_generator=False, use_encryption=None, encryption_key=None)[source]¶
Memoize the results of a callable inside the configured cache.
Data can be encrypted using AES encryption, for safe storage of potentially sensitive information or state. This adds to the processing time slightly, but can be important for cache keys containing sensitive information or that impact access control, particularly in the event that the cache is compromised or shared between services.
Changed in version 3.0: Added support for encrypting keys and data through the
use_encryptionandencryption_keyarguments.Changed in version 2.2.4: Added support for non-iterable value types.
- Parameters:
key (
unicode) – The key to use in the cache.lookup_callable (
callable) – A callable to execute in the case where the data did not exist in the cache.expiration (
int) – The expiration time for the key, in seconds.force_overwrite (
bool) – IfTrue, the value will always be computed and stored regardless of whether it exists in the cache already.large_data (
bool) – IfTrue, the resulting data will be pickled, gzipped, and (potentially) split up into megabyte-sized chunks. This is useful for very large, computationally intensive hunks of data which we don’t want to store in a database due to the way things are accessed.compress_large_data (
bool) – Compresses the data with zlib compression whenlarge_dataisTrue.use_generator (
bool, deprecated) – This parameter is no longer used and will be removed in Djblets 3.0.use_encryption (
bool, optional) –Whether to use encryption when storing or reading data.
If reading data, and if the data cannot be decrypted with the given key, then the data will be considered to have fallen out of cache.
This defaults to
False, but can be forced on for all cached data by settingsettings.DJBLETS_CACHE_FORCE_ENCRYPTION=True.New in version 3.0.
encryption_key (
bytesorstr, optional) –An explicit AES encryption key to use when passing
use_encryption=True.This defaults the value in
settings.DJBLETS_CACHE_DEFAULT_ENCRYPTION_KEY, if set, or to the default AES encryption key for the server as provided bydjblets.secrets.crypto.get_default_aes_encryption_key().New in version 3.0.
- Returns:
The cached data, or the result of
lookup_callableif uncached.- Return type:
- make_cache_key(key, use_encryption=None, encryption_key=None)[source]¶
Create a cache key guaranteed to avoid conflicts and size limits.
The cache key will be prefixed by the site’s domain, and will be changed to an SHA256 hash if it’s larger than the maximum key size or contains characters not compatible with the cache backend.
Changed in version 3.0:
Added support for encrypting keys through the
use_encryptionandencryption_keyarguments.Changed the hash format for keys to use SHA256 instead of MD5. This will invalidate all old keys in cache, but reduces chances of collision.
Keys will automatically use the hash format if they contain characters unsupported by the cache backend.
The return type is now
str, to generate keys that are more suitable for modern versions of Django.
- Parameters:
key (
str) – The base key to generate a cache key from.use_encryption (
bool, optional) –Whether to generate an encrypted key.
This will generate a HMAC digest of the key. There will be no identifying information in the resulting key.
This defaults to
False, but can be forced on for all cached data by settingsettings.DJBLETS_CACHE_FORCE_ENCRYPTION=True.New in version 3.0.
encryption_key (
bytes, optional) –An explicit AES encryption key to use when passing
use_encryption=True.This defaults the value in
settings.DJBLETS_CACHE_DEFAULT_ENCRYPTION_KEY, if set, or to the default AES encryption key for the server as provided bydjblets.secrets.crypto.get_default_aes_encryption_key().New in version 3.0.
- Returns:
A cache key suitable for use with the cache backend.
- Return type: