summaryrefslogtreecommitdiffstats
path: root/src/server/image.c
Commit message (Collapse)AuthorAgeFilesLines
* [SERVER] Log errno if opening image file failsSimon Rettberg2023-06-301-3/+5
|
* [SERVER] Don't establish uplink when loading image on startupSimon Rettberg2022-09-131-1/+1
| | | | | | | Bringing up a proxy that has been offline for some time will trigger lots of loads and replication on other proxies when booting up again. Just wait until a client actually needs an image before establishing an uplink connection.
* [SERVER] Honor uplinkTimeout directly when connecting to alt-serverSimon Rettberg2021-05-101-3/+2Star
|
* [SERVER] Fix warningsSimon Rettberg2020-11-231-3/+3
|
* [SERVER] Update nextSave timestamp at start of functionSimon Rettberg2020-10-291-4/+6
| | | | | | | | | saveLoadAllCacheMaps() is called frequently, and a 'full' run can take some time. If we only update the nextSave timestamp when we're done, we might already have a concurrent call to the function, which will also do a 'full' run, since the timestamp is not updated yet. This doesn't break anything, but leads to even more disk activity, which is probably already high, given that the previous run is not done yet.
* [BUILD] rewrite CMake build system to track changes of source filesManuel Bentele2020-10-161-3/+3
| | | | | | | | | | This change restructures the source code directories, separates shared form non-shared application code and adds CMake dependencies. These dependencies allow the tracking of changes and trigger a rebuild of those build targets where changed files are involved. WARNING: Note that the support of the DNBD3_SERVER_AFL build option is not supported yet. Thus, the option should be never turned on.
* [SERVER] Fix rid 0 proxy lookup if local version is newerSimon Rettberg2020-08-141-5/+15
| | | | | There was a logic bug that would favor a remotely looked up image rid, even if we already found a higher revision locally.
* [SERVER] Add FUSE modeSimon Rettberg2020-07-281-1/+1
| | | | | Still needs some cleanup and optimizations, variable naming sucks, comments, etc.
* [SERVER] Fix: NULL pointer access in saveLoadAllCacheMaps()Simon Rettberg2020-07-211-0/+2
| | | | Entries in _images array might ne NULL
* [SERVER] Fix: No replication if autoFreeDiskSpaceDelay is disabledSimon Rettberg2020-06-301-4/+12
|
* [SERVER] Check local and remote for updates on rid == 0Simon Rettberg2020-03-311-4/+8
|
* [SERVER] image_ensureDiskSpace should only deletes proxied imagesSimon Rettberg2020-03-201-18/+19
|
* [SERVER] Remember atime in .meta fileSimon Rettberg2020-03-201-62/+136
|
* [SERVER] Forbid hidden files when scanning image dirSimon Rettberg2020-03-201-1/+2
|
* [SERVER] Fix warnings, add assertionsSimon Rettberg2020-03-201-2/+5
|
* [SERVER] Add name param to threadpool_runSimon Rettberg2020-03-191-0/+2
|
* [SERVER] Rewrite uplink queue handlingSimon Rettberg2020-03-131-2/+1Star
| | | | | | - Now uses linked lists instead of huge array - Does prefetch data on client requests - Can have multiple replication requests in-flight
* [SERVER] Fix data typeSimon Rettberg2020-03-091-2/+2
|
* [SERVER] Add printf macro for image (name:rid as %s:%d)Simon Rettberg2020-03-061-18/+14Star
|
* [SERVER] Handle "warn unused result" casesSimon Rettberg2020-03-061-2/+6
|
* [SERVER] Reload cache maps periodically for local imagesSimon Rettberg2020-03-061-46/+83
| | | | | | If an image is incomplete, but has no upstream server that can be used for replication, reload the cache map from disk periodically, in case some other server instance is writing to the image.
* [SERVER] Add timer task for saving cache mapsSimon Rettberg2020-03-041-1/+135
| | | | | | | | | Cache maps will now be saved periodically, but only if either they have a "dirty" bit set, which happens if any bits in the map get cleared again (due to corruption), or if new data has been replicated from an uplink server. This either means at least one byte received and 5 minutes have passed, or at least 500MB have been downloaded. The timer currently runs every 20 seconds.
* [SERVER] Likewise, get rid of same loops in client handlerSimon Rettberg2020-03-041-16/+16
|
* [SERVER] Get rid of two loops in image_updateCacheMapSimon Rettberg2020-03-031-22/+18Star
|
* [SERVER] Expose image->problem bools as bitmask in RPC json dataSimon Rettberg2020-03-031-2/+11
|
* [SERVER] Remove "working" flag, introduce fine-grained flagsSimon Rettberg2020-03-031-93/+100
| | | | | | | | Tracking the "working" state of images using one boolean is insufficient regarding the different ways in which providing an image can fail. Introduce separate flags for different conditions, like "file not readable", "file not writable", "no uplink server available", "file content has changed".
* [SERVER] Introduce ignoreAllocErrorsSimon Rettberg2020-02-241-2/+7
| | | | | If enabled, a failed fallocate will not abort image replication, but retry with sparse mode.
* [SERVER] Lookup image on storage even in proxy modeSimon Rettberg2020-01-281-8/+11
| | | | | | | In proxy mode, when rid 0 is requested, we now first query our uplink servers for the latest revision and if this fails, like in non-proxy mode, we'll see what the latest version on disk is.
* [SERVER] Fix checking images without cache mapSimon Rettberg2019-10-291-7/+11
|
* [SERVER] Make buffer when reading for crc check largerSimon Rettberg2019-09-111-1/+1
|
* [SERVER] Make integrity checks on startup asyncSimon Rettberg2019-09-101-25/+24Star
|
* [SERVER] rpc: Add cachemap featureSimon Rettberg2019-09-061-0/+16
|
* [SERVER] Introduce autoFreeDiskSpaceDelaySimon Rettberg2019-09-051-6/+8
| | | | | | | | This setting allows you to control the formerly hard-coded timeout of 10 hours before a proxy would start deleting old images in order to free up space for new images. Setting it to -1 entirely disables automatic deletion, in case you have an external process for freeing up disk space.
* [SERVER] Support limiting alt-servers to specific namespaceSimon Rettberg2019-09-041-1/+1
| | | | | | Not really namespace but simple string matching for the image path. Path is matched from start with no support for glob or regex, so usually you want to have a trailing '/' to limit to certain directories.
* [SERVER] Fix indentationSimon Rettberg2019-09-031-4/+4
|
* [SERVER] Fix image_updateCachemap()Simon Rettberg2019-09-031-4/+8
|
* [SERVER] No uplink_init when checking working state; improve loggingSimon Rettberg2019-08-301-8/+10
|
* [SERVER] Use weakref for cache mapsSimon Rettberg2019-08-291-76/+132
| | | | | | Gets rid of a bunch of locking, especially the hot path in net.c where clients are requesting data. Many clients unsing the same incomplete image previously created a bottleneck here.
* [SERVER] Reintroduce check whether readFd is actually != -1Simon Rettberg2019-08-281-1/+3
|
* [SERVER] Make signal handling more POSIXSimon Rettberg2019-08-281-8/+2Star
| | | | | | | | | | | | | | | According to POSIX, a signal sent to a PID can be delivered to an arbitrary thread of that process that hasn't the signal blocked. This seens to never happen on Linux, but would mess things up since the code expected the main signal handler to only be executed by the main thread. This should now be fixed by examining the destination PID of the signal as well as the ID of the thread currently running the signal handler. If we notice the signal wasn't sent by our own PID and the handler is not currently run by the main thread, we re-send the signal to the main thread. Otherwise, if the signal was sent by our own PID but the handler is not run in the main thread, do nothing. This way we can use pthread_kill() to wake up threads that might be stuck in a blocking syscall when it's time to shut down.
* [SERVER] Remove old commentsSimon Rettberg2019-08-281-30/+0Star
|
* [SERVER] Handle closeUnusedFd via timerSimon Rettberg2019-08-281-17/+19
|
* [SERVER] Use reference counting for uplinkSimon Rettberg2019-08-271-22/+17Star
| | | | First step towards less locking for proxy mode
* [SERVER] Get rid of alt-servers thread, per-uplink rtt historySimon Rettberg2019-08-221-3/+3
| | | | | | | | | | Alt-Server checks are now run using the threadpool, so we don't need a queue and dedicated thread anymore. The rtt history is now kept per uplink, so many uplinks won't overwhelm the history, making its time window very short. Also the fail counter is now split up; a global one for when the server actually isn't reachable, a local (per-uplink) one for when the server is reachable but doesn't serve the requested image.
* [SERVER] Add struct representing active connection to uplink serverSimon Rettberg2019-08-181-1/+1
|
* [SERVER] Better lock debugging: Always check lock orderSimon Rettberg2019-08-071-5/+5
| | | | | | Lock order is predefined in locks.h. Immediately bail out if a lock with lower priority is obtained while the same thread already holds one with higher priority.
* [SERVER] Make image->users atomic and get rid of some lockingSimon Rettberg2019-08-021-52/+39Star
| | | | | | | | With this change it should be safe to read the users count of an image without locking first, assuming you already have a reference on the image or are otherwise sure it cannot be freed, i.e. in an active uplink. Updating users, or checking whether it's 0 in order to free the image should only be done while holding the imageListLock.
* [SERVER] Turn all spinlocks into mutexesSimon Rettberg2019-07-261-97/+99
| | | | | | | | Just assume sane platforms offer smart mutexes that have a fast-path with spinlocks internally for locks that have little to no congestion. In all other cases, mutexes should perform better anyways.
* [SERVER] Export image idle time in json rpcSimon Rettberg2019-01-311-3/+6
| | | | Counter in seconds for how long this image hasn't been used.
* [SERVER] Use O_DIRECT for integrity checksSimon Rettberg2018-07-041-4/+12
| | | | | | | | | | | | | | The idea is that for full image checks, we don't want to pollute the fs cache with gigabytes of data that won't be needed again soon. This would certainly hurt performance on servers that dont have hundreds of GBs of RAM. For single block checks during replication this has the advantage that we don't check the block in memory before it hit the disk once, but actually flush the data to disk, then remove it from the page cache, and only then read it again, from disk. TODO: Might be worth making this a config option
generated by cgit v1.2.3-55-g7522 (git 2.39.0) at 2024-05-03 20:08:17 +0200