summaryrefslogtreecommitdiffstats
path: root/src/server/globals.h
Commit message (Collapse)AuthorAgeFilesLines
* [SERVER] Add minRequestSize: Enlarge relayed requestsSimon Rettberg2021-05-111-0/+6
| | | | | | | | | | Any request from a client being relayed to an uplink server will have its size extended to this value. It will also be applied to background replication requests, if the BGR mode is FULL. As request coalescing is currently very primitive, this setting should usually be left diabled, and bgrWindowSize used instead, if appropriate. If you enable this, set it to something large (1M+), or it might have adverse effects.
* [SERVER] Make prefetching synchronousSimon Rettberg2021-04-141-3/+12
| | | | | | | | | There is a race condition where we process the next request from the same client faster than the OS will schedule the async prefetch job, rendering it a NOOP in the best case (request ranges match) or fetching redundant data from the upstream server (prefetch range is larger than actual request by client). Make prefetching synchronous to prevent this race condition.
* [BUILD] rewrite CMake build system to track changes of source filesManuel Bentele2020-10-161-4/+4
| | | | | | | | | | This change restructures the source code directories, separates shared form non-shared application code and adds CMake dependencies. These dependencies allow the tracking of changes and trigger a rebuild of those build targets where changed files are involved. WARNING: Note that the support of the DNBD3_SERVER_AFL build option is not supported yet. Thus, the option should be never turned on.
* [SERVER] Add FUSE modeSimon Rettberg2020-07-281-2/+5
| | | | | Still needs some cleanup and optimizations, variable naming sucks, comments, etc.
* [SERVER] Remember atime in .meta fileSimon Rettberg2020-03-201-0/+1
|
* [SERVER] Fix warnings, add assertionsSimon Rettberg2020-03-201-1/+1
|
* [SERVER] Rewrite uplink queue handlingSimon Rettberg2020-03-131-10/+25
| | | | | | - Now uses linked lists instead of huge array - Does prefetch data on client requests - Can have multiple replication requests in-flight
* [SERVER] Add printf macro for image (name:rid as %s:%d)Simon Rettberg2020-03-061-0/+1
|
* [SERVER] Reload cache maps periodically for local imagesSimon Rettberg2020-03-061-1/+2
| | | | | | If an image is incomplete, but has no upstream server that can be used for replication, reload the cache map from disk periodically, in case some other server instance is writing to the image.
* [SERVER] Add timer task for saving cache mapsSimon Rettberg2020-03-041-1/+2
| | | | | | | | | Cache maps will now be saved periodically, but only if either they have a "dirty" bit set, which happens if any bits in the map get cleared again (due to corruption), or if new data has been replicated from an uplink server. This either means at least one byte received and 5 minutes have passed, or at least 500MB have been downloaded. The timer currently runs every 20 seconds.
* [SERVER] Expose image->problem bools as bitmask in RPC json dataSimon Rettberg2020-03-031-2/+2
|
* [SERVER] Also add a flag for uplink queue overloadSimon Rettberg2020-03-031-1/+2
|
* [SERVER] Remove "working" flag, introduce fine-grained flagsSimon Rettberg2020-03-031-1/+6
| | | | | | | | Tracking the "working" state of images using one boolean is insufficient regarding the different ways in which providing an image can fail. Introduce separate flags for different conditions, like "file not readable", "file not writable", "no uplink server available", "file content has changed".
* [SERVER] Introduce ignoreAllocErrorsSimon Rettberg2020-02-241-0/+6
| | | | | If enabled, a failed fallocate will not abort image replication, but retry with sparse mode.
* [SERVER] rpc: Add cachemap featureSimon Rettberg2019-09-061-1/+1
|
* [SERVER] Introduce autoFreeDiskSpaceDelaySimon Rettberg2019-09-051-0/+7
| | | | | | | | This setting allows you to control the formerly hard-coded timeout of 10 hours before a proxy would start deleting old images in order to free up space for new images. Setting it to -1 entirely disables automatic deletion, in case you have an external process for freeing up disk space.
* [SERVER] Support limiting alt-servers to specific namespaceSimon Rettberg2019-09-041-0/+8
| | | | | | Not really namespace but simple string matching for the image path. Path is matched from start with no support for glob or regex, so usually you want to have a trailing '/' to limit to certain directories.
* [SERVER] Use weakref for cache mapsSimon Rettberg2019-08-291-2/+8
| | | | | | Gets rid of a bunch of locking, especially the hot path in net.c where clients are requesting data. Many clients unsing the same incomplete image previously created a bottleneck here.
* [SERVER] Make signal handling more POSIXSimon Rettberg2019-08-281-0/+1
| | | | | | | | | | | | | | | According to POSIX, a signal sent to a PID can be delivered to an arbitrary thread of that process that hasn't the signal blocked. This seens to never happen on Linux, but would mess things up since the code expected the main signal handler to only be executed by the main thread. This should now be fixed by examining the destination PID of the signal as well as the ID of the thread currently running the signal handler. If we notice the signal wasn't sent by our own PID and the handler is not currently run by the main thread, we re-send the signal to the main thread. Otherwise, if the signal was sent by our own PID but the handler is not run in the main thread, do nothing. This way we can use pthread_kill() to wake up threads that might be stuck in a blocking syscall when it's time to shut down.
* [SERVER] Use reference counting for uplinkSimon Rettberg2019-08-271-1/+3
| | | | First step towards less locking for proxy mode
* [SERVER] Get rid of alt-servers thread, per-uplink rtt historySimon Rettberg2019-08-221-15/+26
| | | | | | | | | | Alt-Server checks are now run using the threadpool, so we don't need a queue and dedicated thread anymore. The rtt history is now kept per uplink, so many uplinks won't overwhelm the history, making its time window very short. Also the fail counter is now split up; a global one for when the server actually isn't reachable, a local (per-uplink) one for when the server is reachable but doesn't serve the requested image.
* [SERVER] Add struct representing active connection to uplink serverSimon Rettberg2019-08-181-9/+5Star
|
* [SERVER] uplink: More consistent type/variable namingSimon Rettberg2019-08-181-3/+9
| | | | | * Change link to uplink everywhere * dnbd3_connection_t -> dnbd3_uplink_t
* [SERVER] Use more _AtomicSimon Rettberg2019-08-071-3/+3
|
* [SERVER] uplink: Improve attaching to existing requestsSimon Rettberg2019-08-061-12/+0Star
| | | | | Allow attaching in ULR_PROCESSING state, leave lower slots empty to increase chances attaching to ULR_PROCESSING.
* [SERVER] Atomicize some global flagsSimon Rettberg2019-08-031-1/+1
|
* [SERVER] Make image->users atomic and get rid of some lockingSimon Rettberg2019-08-021-1/+1
| | | | | | | | With this change it should be safe to read the users count of an image without locking first, assuming you already have a reference on the image or are otherwise sure it cannot be freed, i.e. in an active uplink. Updating users, or checking whether it's 0 in order to free the image should only be done while holding the imageListLock.
* [SERVER] Turn all spinlocks into mutexesSimon Rettberg2019-07-261-4/+4
| | | | | | | | Just assume sane platforms offer smart mutexes that have a fast-path with spinlocks internally for locks that have little to no congestion. In all other cases, mutexes should perform better anyways.
* [SERVER] Add pretendClient config optionSimon Rettberg2019-07-231-0/+7
| | | | | | This makes the server not set the FLAGS8_SERVER flag when establishing an uplink connection. Useful mostly for running a proxy on localhost for local caching.
* [SERVER] uplink: Relay request in client's thread if possibleSimon Rettberg2019-07-231-0/+1
| | | | | Early benchmarking shows that this is faster, since we don't require another thread to wake up just to send out the request.
* [SERVER] Don't keep an uplink connection established foreverSimon Rettberg2019-01-311-1/+1
| | | | | | In case we don't use background replication a connection to an uplink server can potentially stay around forever. This in turn would prevent the uplink server from freeing the image as it appears to be in use.
* [SERVER] Use atomic vars for uplink byte countersSimon Rettberg2018-08-131-2/+1Star
|
* [SERVER] Use atomic var for (total)bytesSent countersSimon Rettberg2018-08-081-7/+5Star
| | | | | | Gets rid of the lastBytesSent field as well as the stats lock per client. Cleaned and split up the messy net_clientsToJson function while at it.
* [SERVER] Reload server config on SIGHUPSimon Rettberg2018-07-101-19/+20
| | | | | | | Most config settings can now be changed at runtime using SIGHUP. This currently excludes the basePath, listenPort, and the client and image count limits, as well as vmdkLegacyMode.
* [SERVER] Introduce backgroundReplication=hashblockSimon Rettberg2018-07-091-1/+5
| | | | | | | | This is a compromise; if you want to validate replicated data fairly quickly, using this option will make background replication only kick in when there's a "dirty" 16M block, i.e. some blocks within a 16M block are cached locally, but not all. Completing the block makes it possible to validate its CRC32 checksum.
* [SERVER] Refactor uplink/cache handling, improve crc checkingSimon Rettberg2018-07-041-2/+3
| | | | | | | | | | | | | The cacheFd is now moved to the uplink data structure and will only be handled by the uplink thread. The integrity checker now supports checking all blocks of an image. This will be triggered automatically whenever a check for a single block failed. Also, if a crc check on startup fails, the image won't be discarded anymore, but rather a full check will be initiated. Furthermore, when calling image_updateCacheMap() on an image that was previously complete, the cache map will now be re-initialized, and a new uplink connection created.
* [SERVER] Add bgrMinClients: Thresold to control when BGR startsSimon Rettberg2018-04-121-0/+5
| | | | | Background replication will not kick in if there aren't at least that many clients connected.
* [SERVER] Experimental support for sparse files in proxy modeSimon Rettberg2018-03-151-0/+14
| | | | | | | | | | | | Will not preallocate images in this mode. Old images are only deleted if the disk is full, determined by write() calls to the cache file yielding ENOSPC or EDQUOT. In such a case, the least recently used image(s) will be deleted to free up at least 256MiB, and then the write() call will be repeated. This *should* work somewhat reliably unless the cache partition is ridiculously small. Performance might suffer a little, and disk fragmentation might occur much faster than in prealloc mode. Testing is needed.
* [SERVER] Add multiple config options for limiting stuffSimon Rettberg2017-11-081-0/+29
| | | | | maxClients, maxImages, maxPayload, maxReplicationSize Refs #3231
* [SERVER] Support finer control over replication when a proxy connects to a proxySimon Rettberg2017-11-021-0/+7
| | | | | | | Introduce new flag in "select image" message to tell the uplink server whether we have background replication enabled or not. Also reject a connecting proxy if the connecting proxy uses BGR but we don't, as this would basically force the image to be replicated locally too.
* [SERVER] Improve handling of byte stats countersSimon Rettberg2017-10-251-3/+4
| | | | Less writes to variables, more up-to-date values for uplinks.
* [SERVER] Fix types or add explicit casts everywhere we might have type ↵Simon Rettberg2017-10-241-4/+4
| | | | conversion problems
* [SERVER] Use monotonic clock for measuring timeSimon Rettberg2017-10-191-5/+9
| | | | | Introduces new shared source unit timing.[ch] Closes #3214
* [SERVER] Try to connect to different server if proxy cycle is detectedSimon Rettberg2017-10-171-0/+1
|
* [*] Support hop-counting in request header, protocol version 3Simon Rettberg2017-10-171-3/+6
| | | | | | | | | | | | | | We steal 8 bits from the request offset to count hops when requests get relayed by proxies. This still leaves plenty of bits for the offset (56 bits, supporting images of up to 72 petabytes). This is used to detect proxy cycles. The algorithm is not perfect but should prevent endless relays of the same request. This is backwards compatible to old clients and servers, as the server only ever sets the hopcount in relayed requests if the upstream server is using protocol version 3 or newer, and clients are automatically upwards compatible as there is practically no image larger than 74PB, so the newly introduced hop count field is always 0 even in requests from old clients.
* [SERVER] Implement HTTP RPC that supports different queries and ACLSimon Rettberg2017-10-011-15/+5Star
| | | | | | | | | - ACL is defined in new file rpc.acl - Queries are still WIP, for now something like /query?q=stats&q=images /query?q=clients works, although the parsing is still ugly - Also supports HTTP keep-alive
* [*] Continue splitting #defines to clientconfig.h etc.Simon Rettberg2017-09-021-1/+1
|
* [SERVER] Split server-only defs from config.h to serverconfig.hSimon Rettberg2017-08-261-0/+7
|
* [SHARED] signal.h -> fdsignal.hSimon Rettberg2016-02-101-1/+1
|
* First steps in make signals more abstract from the underlying mechanism; ↵Simon Rettberg2016-02-051-1/+2
| | | | | | | | | | replace epoll with poll. We now don't assume that a signal equals a single fd (eventfd on Linux). The next step would be to create a version of signal.c that uses a pipe internally, so it can be used on other platforms, like *BSD. This is also the reason epoll was replaced with poll in uplink.c
generated by cgit v1.2.3-55-g7522 (git 2.39.0) at 2024-05-14 00:07:33 +0200