| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
|
|
|
|
| |
This setting allows you to control the formerly hard-coded timeout of 10
hours before a proxy would start deleting old images in order to free up
space for new images. Setting it to -1 entirely disables automatic
deletion, in case you have an external process for freeing up disk
space.
|
|
|
|
|
|
| |
Not really namespace but simple string matching for the image path. Path
is matched from start with no support for glob or regex, so usually you
want to have a trailing '/' to limit to certain directories.
|
|
|
|
|
|
| |
Gets rid of a bunch of locking, especially the hot path in net.c where
clients are requesting data. Many clients unsing the same incomplete
image previously created a bottleneck here.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
According to POSIX, a signal sent to a PID can be delivered to an
arbitrary thread of that process that hasn't the signal blocked. This
seens to never happen on Linux, but would mess things up since the code
expected the main signal handler to only be executed by the main thread.
This should now be fixed by examining the destination PID of the signal
as well as the ID of the thread currently running the signal handler. If
we notice the signal wasn't sent by our own PID and the handler is not
currently run by the main thread, we re-send the signal to the main
thread. Otherwise, if the signal was sent by our own PID but the handler
is not run in the main thread, do nothing. This way we can use
pthread_kill() to wake up threads that might be stuck in a blocking
syscall when it's time to shut down.
|
|
|
|
| |
First step towards less locking for proxy mode
|
|
|
|
|
|
|
|
|
|
| |
Alt-Server checks are now run using the threadpool, so we don't need a
queue and dedicated thread anymore. The rtt history is now kept per
uplink, so many uplinks won't overwhelm the history, making its time
window very short.
Also the fail counter is now split up; a global one for when the server
actually isn't reachable, a local (per-uplink) one for when the server
is reachable but doesn't serve the requested image.
|
| |
|
|
|
|
|
| |
* Change link to uplink everywhere
* dnbd3_connection_t -> dnbd3_uplink_t
|
| |
|
|
|
|
|
| |
Allow attaching in ULR_PROCESSING state, leave lower slots empty
to increase chances attaching to ULR_PROCESSING.
|
| |
|
|
|
|
|
|
|
|
| |
With this change it should be safe to read the users count of an image
without locking first, assuming you already have a reference on the
image or are otherwise sure it cannot be freed, i.e. in an active
uplink. Updating users, or checking whether it's 0 in order to free the
image should only be done while holding the imageListLock.
|
|
|
|
|
|
|
|
| |
Just assume sane platforms offer smart mutexes
that have a fast-path with spinlocks internally
for locks that have little to no congestion.
In all other cases, mutexes should perform better
anyways.
|
|
|
|
|
|
| |
This makes the server not set the FLAGS8_SERVER flag
when establishing an uplink connection. Useful mostly
for running a proxy on localhost for local caching.
|
|
|
|
|
| |
Early benchmarking shows that this is faster, since we don't
require another thread to wake up just to send out the request.
|
|
|
|
|
|
| |
In case we don't use background replication a connection to an uplink
server can potentially stay around forever. This in turn would prevent
the uplink server from freeing the image as it appears to be in use.
|
| |
|
|
|
|
|
|
| |
Gets rid of the lastBytesSent field as well as the stats lock per
client. Cleaned and split up the messy net_clientsToJson function while
at it.
|
|
|
|
|
|
|
| |
Most config settings can now be changed at runtime
using SIGHUP. This currently excludes the basePath,
listenPort, and the client and image count limits,
as well as vmdkLegacyMode.
|
|
|
|
|
|
|
|
| |
This is a compromise; if you want to validate replicated data fairly
quickly, using this option will make background replication only kick in
when there's a "dirty" 16M block, i.e. some blocks within a 16M block
are cached locally, but not all. Completing the block makes it possible
to validate its CRC32 checksum.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The cacheFd is now moved to the uplink data structure and will
only be handled by the uplink thread.
The integrity checker now supports checking all blocks of an
image. This will be triggered automatically whenever a check for
a single block failed.
Also, if a crc check on startup fails, the image won't be discarded
anymore, but rather a full check will be initiated.
Furthermore, when calling image_updateCacheMap() on an image that
was previously complete, the cache map will now be re-initialized,
and a new uplink connection created.
|
|
|
|
|
| |
Background replication will not kick in if there aren't at least
that many clients connected.
|
|
|
|
|
|
|
|
|
|
|
|
| |
Will not preallocate images in this mode. Old images are only
deleted if the disk is full, determined by write() calls to
the cache file yielding ENOSPC or EDQUOT. In such a case,
the least recently used image(s) will be deleted to free
up at least 256MiB, and then the write() call will be repeated.
This *should* work somewhat reliably unless the cache partition
is ridiculously small. Performance might suffer a little, and
disk fragmentation might occur much faster than in prealloc
mode. Testing is needed.
|
|
|
|
|
| |
maxClients, maxImages, maxPayload, maxReplicationSize
Refs #3231
|
|
|
|
|
|
|
| |
Introduce new flag in "select image" message to tell the uplink server
whether we have background replication enabled or not. Also reject
a connecting proxy if the connecting proxy uses BGR but we don't, as this
would basically force the image to be replicated locally too.
|
|
|
|
| |
Less writes to variables, more up-to-date values for uplinks.
|
|
|
|
| |
conversion problems
|
|
|
|
|
| |
Introduces new shared source unit timing.[ch]
Closes #3214
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We steal 8 bits from the request offset to count hops when requests
get relayed by proxies. This still leaves plenty of bits for the
offset (56 bits, supporting images of up to 72 petabytes).
This is used to detect proxy cycles. The algorithm is not perfect
but should prevent endless relays of the same request.
This is backwards compatible to old clients and servers, as the server
only ever sets the hopcount in relayed requests if the upstream server
is using protocol version 3 or newer, and clients are automatically
upwards compatible as there is practically no image larger than 74PB,
so the newly introduced hop count field is always 0 even in requests
from old clients.
|
|
|
|
|
|
|
|
|
| |
- ACL is defined in new file rpc.acl
- Queries are still WIP, for now something like
/query?q=stats&q=images
/query?q=clients
works, although the parsing is still ugly
- Also supports HTTP keep-alive
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
replace epoll with poll.
We now don't assume that a signal equals a single fd (eventfd on Linux).
The next step would be to create a version of signal.c that uses a pipe
internally, so it can be used on other platforms, like *BSD.
This is also the reason epoll was replaced with poll in uplink.c
|
|
|
|
|
|
|
|
|
|
| |
This was a wrong decision made long time ago, and it's broken in
certain scenarios (eg. two servers serving from same NFS mount).
Also it's of limited use anyways since it only supportes ASCII and
would ignore umlauts, so blöd and BLÖD would still be considered
two different images.
So if you relied on this "feature" in any way, be careful when
updating.
|
| |
|
|
|
|
|
|
|
| |
A run with gprof revealed that background replication is a huge CPU hog.
The block selection was very slow and has been improved a lot.
Minor improvements were made to other functions that scan the cache map
of an image and are thus relatively slow.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Now that we can automatically load unknown images from disk on request,
it makes sense to remove non-working images from the image list. On
future requests, we will look for them on disk again, which is nice
in case of temporary storage hickups.
Also, some more ore less related locking has been refined (loading images,
replicating images)
|
| |
|
| |
|
|
|
|
| |
names of the byte count fields in RPC consistent
|
| |
|
|
|
|
| |
Some minor debugging code, mostly commented out
|
| |
|
|
|
|
| |
condition in uplink_init
|
|
|
|
|
|
|
| |
compare and some unused params)
Also allow newline at end of alt-servers file without complaining about it
not being a valid alt server definition.
|
|
|
|
| |
handling signals even though we block them
|