| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
| |
This setting allows you to control the formerly hard-coded timeout of 10
hours before a proxy would start deleting old images in order to free up
space for new images. Setting it to -1 entirely disables automatic
deletion, in case you have an external process for freeing up disk
space.
|
|
|
|
|
|
| |
Not really namespace but simple string matching for the image path. Path
is matched from start with no support for glob or regex, so usually you
want to have a trailing '/' to limit to certain directories.
|
| |
|
| |
|
| |
|
|
|
|
|
|
| |
Gets rid of a bunch of locking, especially the hot path in net.c where
clients are requesting data. Many clients unsing the same incomplete
image previously created a bottleneck here.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
According to POSIX, a signal sent to a PID can be delivered to an
arbitrary thread of that process that hasn't the signal blocked. This
seens to never happen on Linux, but would mess things up since the code
expected the main signal handler to only be executed by the main thread.
This should now be fixed by examining the destination PID of the signal
as well as the ID of the thread currently running the signal handler. If
we notice the signal wasn't sent by our own PID and the handler is not
currently run by the main thread, we re-send the signal to the main
thread. Otherwise, if the signal was sent by our own PID but the handler
is not run in the main thread, do nothing. This way we can use
pthread_kill() to wake up threads that might be stuck in a blocking
syscall when it's time to shut down.
|
| |
|
| |
|
|
|
|
| |
First step towards less locking for proxy mode
|
|
|
|
|
|
|
|
|
|
| |
Alt-Server checks are now run using the threadpool, so we don't need a
queue and dedicated thread anymore. The rtt history is now kept per
uplink, so many uplinks won't overwhelm the history, making its time
window very short.
Also the fail counter is now split up; a global one for when the server
actually isn't reachable, a local (per-uplink) one for when the server
is reachable but doesn't serve the requested image.
|
| |
|
|
|
|
|
|
| |
Lock order is predefined in locks.h. Immediately bail out if a lock with
lower priority is obtained while the same thread already holds one with
higher priority.
|
|
|
|
|
|
|
|
| |
With this change it should be safe to read the users count of an image
without locking first, assuming you already have a reference on the
image or are otherwise sure it cannot be freed, i.e. in an active
uplink. Updating users, or checking whether it's 0 in order to free the
image should only be done while holding the imageListLock.
|
|
|
|
|
|
|
|
| |
Just assume sane platforms offer smart mutexes
that have a fast-path with spinlocks internally
for locks that have little to no congestion.
In all other cases, mutexes should perform better
anyways.
|
|
|
|
| |
Counter in seconds for how long this image hasn't been used.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The idea is that for full image checks, we don't want to
pollute the fs cache with gigabytes of data that won't be
needed again soon. This would certainly hurt performance
on servers that dont have hundreds of GBs of RAM.
For single block checks during replication this has the
advantage that we don't check the block in memory before
it hit the disk once, but actually flush the data to disk,
then remove it from the page cache, and only then read it
again, from disk.
TODO: Might be worth making this a config option
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The cacheFd is now moved to the uplink data structure and will
only be handled by the uplink thread.
The integrity checker now supports checking all blocks of an
image. This will be triggered automatically whenever a check for
a single block failed.
Also, if a crc check on startup fails, the image won't be discarded
anymore, but rather a full check will be initiated.
Furthermore, when calling image_updateCacheMap() on an image that
was previously complete, the cache map will now be re-initialized,
and a new uplink connection created.
|
|
|
|
|
|
|
| |
In scenarios where the proxy is using an NFS server as
storage (for whatever crazy reason) or when the cacheFd
goes bad through e.g. a switchroot, try to re-open it
instead of just disabling caching forever.
|
| |
|
| |
|
|
|
|
|
| |
imageListLock was locked on twice in the call stack, which
is bad if you're using non-recursive locks.
|
| |
|
| |
|
| |
|
|
|
|
| |
uplink if existent
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Will not preallocate images in this mode. Old images are only
deleted if the disk is full, determined by write() calls to
the cache file yielding ENOSPC or EDQUOT. In such a case,
the least recently used image(s) will be deleted to free
up at least 256MiB, and then the write() call will be repeated.
This *should* work somewhat reliably unless the cache partition
is ridiculously small. Performance might suffer a little, and
disk fragmentation might occur much faster than in prealloc
mode. Testing is needed.
|
|
|
|
|
| |
maxClients, maxImages, maxPayload, maxReplicationSize
Refs #3231
|
| |
|
|
|
|
| |
Refs #3231
|
|
|
|
|
| |
Just as in the fuse client, this will speed things up if
we have several alt-servers in our list which are not reachable.
|
|
|
|
|
|
|
| |
Introduce new flag in "select image" message to tell the uplink server
whether we have background replication enabled or not. Also reject
a connecting proxy if the connecting proxy uses BGR but we don't, as this
would basically force the image to be replicated locally too.
|
|
|
|
| |
...there were quite a few format string errors as it turns out :/
|
| |
|
| |
|
| |
|
|
|
|
|
|
| |
We only used it for CRC-32, so now the source
tree includes a stripped down version of the crc32
code from the zlib project.
|
|
|
|
| |
conversion problems
|
|
|
|
|
| |
Introduces new shared source unit timing.[ch]
Closes #3214
|
|
|
|
| |
false positives
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We steal 8 bits from the request offset to count hops when requests
get relayed by proxies. This still leaves plenty of bits for the
offset (56 bits, supporting images of up to 72 petabytes).
This is used to detect proxy cycles. The algorithm is not perfect
but should prevent endless relays of the same request.
This is backwards compatible to old clients and servers, as the server
only ever sets the hopcount in relayed requests if the upstream server
is using protocol version 3 or newer, and clients are automatically
upwards compatible as there is practically no image larger than 74PB,
so the newly introduced hop count field is always 0 even in requests
from old clients.
|