如何使用klepto在磁盘上使用LRU缓存？(How to use LRU caching on disk with klepto?)

 我正在尝试使用klepto来执行LRU缓存。 我想将缓存存储到磁盘，目前我正在使用klepto的dir_archive选项。 我写了下面的代码，主要基于klepto测试脚本中的代码： 
 
def mymap(data):
    return hashlib.sha256(data).hexdigest()

class MyLRUCache:
    @lru_cache(cache=dir_archive(cached=False), keymap=mymap, ignore='self', maxsize=5)
    def __call__(self, data)
        return data

    call = __call__

    def store(self, data):
        self.call(data)

    # I would also appreciate a better way to do this, if possible.
    def lookup(self, key):
        return self.call.__cache__()[key]
 
 此代码似乎正常工作，直到缓存达到maxsize 。 在这一点上， lru_cache不是使用LRU来删除单个项目， lru_cache清除整个缓存！ 下面是一段klepto源代码（ https://github.com/uqfoundation/klepto/blob/master/klepto/safe.py ）： 
 
# purge cache
if _len(cache) > maxsize:
    if cache.archived():
        cache.dump()
        cache.clear() 
        queue.clear()
        refcount.clear()
     else: # purge least recently used cache entry
        key = queue_popleft()
        refcount[key] -= 1
        while refcount[key]:
            key = queue_popleft()
            refcount[key] -= 1
        del cache[key], refcount[key]
 
 所以我的问题是，为什么klepto清除“存档”缓存？ 是否可以一起使用lru_cache和dir_archive ？ 
 
 另外，如果我的代码看起来完全是疯狂的，那么我真的很感激我应该写这个的一些示例代码，因为没有太多关于klepto的文档。 
 
 其他注意事项：我也试着用cached=True定义dir_archive 。 内存缓存在达到maxsize大小时仍会被清除，但缓存的内容将转储到此时的归档缓存中。 我有几个问题： 
 
 
 
 内存中的高速缓存只有在达到maxsize大小时才是准确的，在此时它会被擦除。 
 
 
 归档缓存不受maxsize影响。 内存缓存每次达到maxsize大小时，内存缓存中的所有项目都将转储到存档缓存，而不管已存在多少个缓存。 
 
 
 基于第1点和第2点，LRU缓存似乎是不可能的。 
 

I am trying to use klepto to do LRU caching. I would like to store the cache to disk, and am currently using klepto's dir_archive option for this. I have written the following code, largely based on the code in the klepto test scripts:
 
def mymap(data):
    return hashlib.sha256(data).hexdigest()

class MyLRUCache:
    @lru_cache(cache=dir_archive(cached=False), keymap=mymap, ignore='self', maxsize=5)
    def __call__(self, data)
        return data

    call = __call__

    def store(self, data):
        self.call(data)

    # I would also appreciate a better way to do this, if possible.
    def lookup(self, key):
        return self.call.__cache__()[key]
 
This code appears to work fine until the cache reaches maxsize. At that point, instead of using LRU to remove a single item, lru_cache purges the entire cache! Below is the piece of klepto source code that does this (https://github.com/uqfoundation/klepto/blob/master/klepto/safe.py):
 
# purge cache
if _len(cache) > maxsize:
    if cache.archived():
        cache.dump()
        cache.clear() 
        queue.clear()
        refcount.clear()
     else: # purge least recently used cache entry
        key = queue_popleft()
        refcount[key] -= 1
        while refcount[key]:
            key = queue_popleft()
            refcount[key] -= 1
        del cache[key], refcount[key]
 
So my question is, why does klepto purge "archived" caches? Is it possible to use lru_cache and dir_archive together?
 
Also, if my code looks completely nuts, I would really appreciate some sample code of how I should be writing this, since there was not much documentation for klepto.
 
ADDITIONAL NOTES: I also tried defining dir_archive with cached=True. The in-memory cache still gets purged when maxsize is reached, but the contents of the cache are dumped to the archived cache at that point. I have several problems with this:
 
 
 
The in-memory cache is only accurate until maxsize is reached, at which point it is wiped.
 
 
The archived cache is not affected by maxsize. Every time maxsize is reached by the in-memory cache, all items in the in-memory cache are dumped to the archived cache, regardless of how many are already there.
 
 
LRU caching seems impossible based on points 1 and 2.
 

原文：https://stackoverflow.com/questions/32081207