summaryrefslogtreecommitdiffstats
path: root/src/server/uplink.c
Commit message (Collapse)AuthorAgeFilesLines
* [SERVER] Don't keep an uplink connection established foreverSimon Rettberg2019-01-311-38/+70
| | | | | | In case we don't use background replication a connection to an uplink server can potentially stay around forever. This in turn would prevent the uplink server from freeing the image as it appears to be in use.
* [SERVER] uplink: Check for _maxPayload when getting client requestSimon Rettberg2019-01-311-0/+4
| | | | | | It didn't make too much sense that we only checked _maxPayload when the reply arrived; simply don't forward a request where we already know we won't handle the reply.
* [SERVER] Fix last block replicating indefinitely in hasblock modeSimon Rettberg2018-08-141-5/+7
|
* [SERVER] Use atomic vars for uplink byte countersSimon Rettberg2018-08-131-26/+4Star
|
* [SERVER] Use atomic var for (total)bytesSent countersSimon Rettberg2018-08-081-2/+0Star
| | | | | | Gets rid of the lastBytesSent field as well as the stats lock per client. Cleaned and split up the messy net_clientsToJson function while at it.
* [SERVER] Add missing file mode to open() callSimon Rettberg2018-07-261-2/+2
|
* [SERVER] Fix use of wrong constant; increase queue len for hash checkerSimon Rettberg2018-07-101-1/+1
|
* [SERVER] Reduce spam some moreSimon Rettberg2018-07-091-1/+1
|
* [SERVER] Introduce backgroundReplication=hashblockSimon Rettberg2018-07-091-33/+121
| | | | | | | | This is a compromise; if you want to validate replicated data fairly quickly, using this option will make background replication only kick in when there's a "dirty" 16M block, i.e. some blocks within a 16M block are cached locally, but not all. Completing the block makes it possible to validate its CRC32 checksum.
* [SERVER] Don't keep bg replication blocks in fs cacheSimon Rettberg2018-07-051-1/+5
| | | | | | Further improving cache handling, don't keep blocks in cache that have been requested via background replication. It's likely these aren't needed in the near future.
* [SERVER] Always use fsync instead of fdatasyncSimon Rettberg2018-07-051-8/+2Star
| | | | | | | | Now that we support sparse files, using just fdatasync isn't safe anymore. Instead of handling both cases differently just drop fdatasync, the difference has probably been marginal all along anyways.
* [SERVER] Refactor uplink/cache handling, improve crc checkingSimon Rettberg2018-07-041-35/+182
| | | | | | | | | | | | | The cacheFd is now moved to the uplink data structure and will only be handled by the uplink thread. The integrity checker now supports checking all blocks of an image. This will be triggered automatically whenever a check for a single block failed. Also, if a crc check on startup fails, the image won't be discarded anymore, but rather a full check will be initiated. Furthermore, when calling image_updateCacheMap() on an image that was previously complete, the cache map will now be re-initialized, and a new uplink connection created.
* [SERVER] Use likely/unlikely in uplink disk writing loopSimon Rettberg2018-06-251-4/+4
|
* [SERVER] Try to re-open cacheFd if writing failsSimon Rettberg2018-06-251-6/+12
| | | | | | | In scenarios where the proxy is using an NFS server as storage (for whatever crazy reason) or when the cacheFd goes bad through e.g. a switchroot, try to re-open it instead of just disabling caching forever.
* [SERVER] Add bgrMinClients: Thresold to control when BGR startsSimon Rettberg2018-04-121-3/+6
| | | | | Background replication will not kick in if there aren't at least that many clients connected.
* [SERVER] More error handling and logging when caching received data to diskSimon Rettberg2018-04-101-4/+13
|
* [SERVER] Experimental support for sparse files in proxy modeSimon Rettberg2018-03-151-1/+10
| | | | | | | | | | | | Will not preallocate images in this mode. Old images are only deleted if the disk is full, determined by write() calls to the cache file yielding ENOSPC or EDQUOT. In such a case, the least recently used image(s) will be deleted to free up at least 256MiB, and then the write() call will be repeated. This *should* work somewhat reliably unless the cache partition is ridiculously small. Performance might suffer a little, and disk fragmentation might occur much faster than in prealloc mode. Testing is needed.
* [SERVER] altservers: Short timeout during RTT measurement, round request rangeSimon Rettberg2017-11-081-4/+7
| | | | | Rounding to 4k so caching works efficiently This should now close #3231
* [SERVER] Add multiple config options for limiting stuffSimon Rettberg2017-11-081-2/+2
| | | | | maxClients, maxImages, maxPayload, maxReplicationSize Refs #3231
* [SERVER] Use multiConnect() to find uplink for replicationSimon Rettberg2017-11-071-1/+1
| | | | | Just as in the fuse client, this will speed things up if we have several alt-servers in our list which are not reachable.
* [SERVER] uplink: Fix updating of global byte counter, fix incremental updatesSimon Rettberg2017-10-251-2/+7
| | | | | Incremental updating of the global byte counter would only work when background replication is disabled. Fix this.
* [SERVER] Improve handling of byte stats countersSimon Rettberg2017-10-251-4/+13
| | | | Less writes to variables, more up-to-date values for uplinks.
* [SERVER] Get rid of zlib dependencySimon Rettberg2017-10-241-3/+3
| | | | | | We only used it for CRC-32, so now the source tree includes a stripped down version of the crc32 code from the zlib project.
* [SERVER] Fix types or add explicit casts everywhere we might have type ↵Simon Rettberg2017-10-241-5/+5
| | | | conversion problems
* [SERVER] Use monotonic clock for measuring timeSimon Rettberg2017-10-191-16/+20
| | | | | Introduces new shared source unit timing.[ch] Closes #3214
* [SERVER] Try to connect to different server if proxy cycle is detectedSimon Rettberg2017-10-171-2/+15
|
* [*] Support hop-counting in request header, protocol version 3Simon Rettberg2017-10-171-13/+34
| | | | | | | | | | | | | | We steal 8 bits from the request offset to count hops when requests get relayed by proxies. This still leaves plenty of bits for the offset (56 bits, supporting images of up to 72 petabytes). This is used to detect proxy cycles. The algorithm is not perfect but should prevent endless relays of the same request. This is backwards compatible to old clients and servers, as the server only ever sets the hopcount in relayed requests if the upstream server is using protocol version 3 or newer, and clients are automatically upwards compatible as there is practically no image larger than 74PB, so the newly introduced hop count field is always 0 even in requests from old clients.
* [SERVER] Make crclist handling endian-safeSimon Rettberg2017-04-271-0/+1
|
* [SERVER] Fix formattingSimon Rettberg2016-03-221-3/+3
|
* cleanup commitsSebastian2016-02-111-12/+2Star
|
* [SHARED] signal.h -> fdsignal.hSimon Rettberg2016-02-101-1/+1
|
* First steps in make signals more abstract from the underlying mechanism; ↵Simon Rettberg2016-02-051-73/+48Star
| | | | | | | | | | replace epoll with poll. We now don't assume that a signal equals a single fd (eventfd on Linux). The next step would be to create a version of signal.c that uses a pipe internally, so it can be used on other platforms, like *BSD. This is also the reason epoll was replaced with poll in uplink.c
* [SERVER] BREAKING: Get rid of pseudo case-insensitivityv2.1Simon Rettberg2016-02-011-9/+9
| | | | | | | | | | This was a wrong decision made long time ago, and it's broken in certain scenarios (eg. two servers serving from same NFS mount). Also it's of limited use anyways since it only supportes ASCII and would ignore umlauts, so blöd and BLÖD would still be considered two different images. So if you relied on this "feature" in any way, be careful when updating.
* [SERVER] Fix a lot of (mostly harmless) data racesSimon Rettberg2016-01-111-10/+28
|
* [SERVER] Performance: Optimized some functions (gprof)Simon Rettberg2015-12-171-11/+17
| | | | | | | A run with gprof revealed that background replication is a huge CPU hog. The block selection was very slow and has been improved a lot. Minor improvements were made to other functions that scan the cache map of an image and are thus relatively slow.
* [SERVER] image_markComplete now handles locking so we remove() unlockedSimon Rettberg2015-12-171-8/+6Star
|
* [SERVER] More fine grained locking for RPC; better error loggingSimon Rettberg2015-12-161-3/+11
|
* [SERVER] Remove non-working images from list, plus refactoringSimon Rettberg2015-12-151-6/+12
| | | | | | | | | | Now that we can automatically load unknown images from disk on request, it makes sense to remove non-working images from the image list. On future requests, we will look for them on disk again, which is nice in case of temporary storage hickups. Also, some more ore less related locking has been refined (loading images, replicating images)
* [SERVER] Support looking on disk if an unknown image is requestedSimon Rettberg2015-12-121-4/+4
|
* [SERVER] Nullpad images virtually at runtime instead of padding the actual fileSimon Rettberg2015-12-101-5/+5
|
* [SERVER] Update includes for moved log.hSimon Rettberg2015-12-021-1/+1
|
* [FUSE] Mid-refactoring, does not compileSimon Rettberg2015-11-241-1/+1
|
* [FUSE] Start refactoring so we can handle multithread fuseSimon Rettberg2015-11-211-2/+2
|
* [SERVER] Fix integer overflow of global bytesReceived getterSimon Rettberg2015-05-131-1/+1
|
* [Server] Added statistics of images and uptime to json.Stephan Schwaer2015-05-051-9/+3Star
|
* Removed dnbd3_printClients, added clients to statistics json in rpc.Stephan Schwaer2015-05-041-3/+17
|
* [SERVER] Added counters for received and sent bytes.Stephan Schwaer2015-04-271-1/+18
|
* [FUSE] Integrate into cmake build processSimon Rettberg2015-04-241-1/+1
|
* [SERVER] printf -> logadd, make USR2 trigger a log cycle, remove old fake ↵Simon Rettberg2015-02-231-3/+3
| | | | delay feature
* [SERVER] Overhauled loggingSimon Rettberg2015-02-221-32/+32
| | | | | | - Added message type parameter - Log to file and stdout, no more logging in memory - Added options to server.conf to filter which messages show up where
generated by cgit v1.2.3-55-g7522 (git 2.39.0) at 2024-11-13 08:19:11 +0100