summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorMichael Scherle2022-07-05 12:42:30 +0200
committerMichael Scherle2022-07-05 12:42:30 +0200
commit607eb8c0593d2ab11be17b759abcca24167f808c (patch)
tree56b921975b7d7760b8063ddc1f685ba4d3982438
parentadded blockupload dump into status.txt (diff)
downloaddnbd3-607eb8c0593d2ab11be17b759abcca24167f808c.tar.gz
dnbd3-607eb8c0593d2ab11be17b759abcca24167f808c.tar.xz
dnbd3-607eb8c0593d2ab11be17b759abcca24167f808c.zip
corrected typos in cow redme
-rw-r--r--src/fuse/cowDoc/readme.md41
1 files changed, 20 insertions, 21 deletions
diff --git a/src/fuse/cowDoc/readme.md b/src/fuse/cowDoc/readme.md
index c59d091..f707bff 100644
--- a/src/fuse/cowDoc/readme.md
+++ b/src/fuse/cowDoc/readme.md
@@ -39,16 +39,16 @@ Example parameters for creating a new cow session:
## Data structure
-The data structure is split in two main parts. The actual data from the write on the image and its corresponding metadata. Its also important to distinguish between a dnbd3 block which is 4096byte and and cow block which groups 320 dnbd3 blocks together. An cow block has an `cow_block_metadata_t` struct which holds the corresponding meta data. The metadata is used to determine if and block has bin written on, where this block is stored in the data file, when it was last modified and when it was uploaded. But more later.
+The data structure is split in two main parts. The actual data from the write on the image and its corresponding metadata. Its also important to distinguish between a dnbd3 block which is 4096byte and and cow block which groups 320 dnbd3 blocks together. An cow block has an `cow_block_metadata_t` struct which holds the corresponding meta data. The metadata is used to determine if and block has been written on, where this block is stored in the data file, when it was last modified and when it was uploaded. But more later.
### Blockmetadata
![Datastructure](img/Bild1.jpg)
-The data structure for storing metadata about cow blocks contains a Layer 1(L1) and a Layer 2 (L2). L1 contains pointers to the L2's.
+The data structure for storing metadata cow blocks contains a Layer 1(L1) and a Layer 2 (L2). L1 contains pointers to the L2's.
The whole L1 array is initialized at the beginning and cannot be resized, so the size of the L1 array limits the total size of the image.
-The L2's are dynamically created once needed. So at the beginning, all L1 pointer will be null. The L2's are arrays which contains 1024
+The L2's are dynamically created once needed. So at the beginning, all L1 pointers will be null. The L2's are arrays which contain 1024
`cow_block_metadata_t` structs.
```C
@@ -60,7 +60,7 @@ typedef struct cow_block_metadata
atomic_char bitfield[40];
} cow_block_metadata_t;
```
-Each `cow_block_metadata_t` contains a 40 byte so 320 bit bitfield. The bitfield indicates whether the corresponding dnbd3 block contains data or not. For e.g. if the bitfield starts with 01.., the first 4096 contains not data and the next 4096 contain data.
+Each `cow_block_metadata_t` contains a 40 byte so 320 bit bit field. The bit field indicates whether the corresponding dnbd3 block contains data or not. For e.g. if the bit field starts with 01.., the first 4096 contains not data and the next 4096 contain data.
So each `cow_block_metadata_t` stores the metadata of up to 320*4096 byte if all bits are set to 1. The offset is the offset where in the data file is the corresponding data stored. The timeChanged property contains the unix when the block was last modified. It's 0 if it was never modified or if the last changes are already uploaded.
@@ -88,27 +88,26 @@ COW_METADATA_STORAGE_CAPACITY = 320 * 4096
-For an read request, for every 4096byte block it will be checked if the block is already local on the computer (therefore was already written to before).
-If so it will be read from disk, otherwise it will be requested from the dnbd3 server. To increase performance, multiple following blocks that are also local/non local like the block before will be combined to to one larger reads from disc respectively one larger request from the server.
+For an read request, for every 4096byte block it will be checked if the block is already locally on the computer (therefore was already written before). If so it will be read from disk, otherwise it will be requested from the dnbd3 server. To increase performance, multiple following blocks that are also local/non local like the block before will be combined to to one larger reads from disc respectively one larger request from the server.
![readrequest](img/readrequest.svg)
-The graph shown above is somewhat simplified for better visibility. The reads from the server happen async. So it will not be waited for the server to respond, rather the it will move on with the next blocks. As soon as the respond from the server is finished, the data will be written in
+The graph shown above is somewhat simplified for better visibility. The reads from the server happen async. So it will not be waiting for the server to respond, rather the it will move on with the next blocks. As soon as the respond from the server is finished, the data will be written in
the fuse buffer. Each request to the dnbd3 server will increase the `workCounter` variable by one and every time a request is done it will be decreased by one. Once `workCounter` is 0 again, fuse_request will be returned.
Also on the local side, it has to break the loop once the end of an `cow_block_metadata_t` is reached, since the next data offset of the next `cow_block_metadata_t` is very likely not directly after it in the data file.
### Write Request
For the write request, if the start or the end or the end does not align with a multiple of 4096, then the start and/or end block must be padded.
-Because every 4096byte block needs complete data, since if the bit in the bitfield for that block is set, all the data will be read locally.
-To pad the block, if its still in the range of the original image size, the missing bytes will be requested from the dnbd3 server. If its outside of the original image (because the image grown in size) then the missing bytes will be padded with 0.
+Because every 4096byte block needs complete data, since if the bit in the bit field for that block is set, all the data will be read locally.
+To pad the block, if it's still in the range of the original image size, the missing bytes will be requested from the dnbd3 server. If it's outside of the original image (because the image grown in size) then the missing bytes will be padded with 0.
The write request will write compute the corresponding `cow_block_metadata_t` from the offset. If the corresponding `cow_block_metadata_t` does not exist yet it will be created. The data will be written in the data file, at the offset which is stored in the `cow_block_metadata_t`.
-Then the corresponding bit in the bitfields will be set and the `timeChanged` will be updated. If there is more data to write, the next `cow_block_metadata_t` will be computed and the steps above will be repeated.
+Then the corresponding bit in the bit fields will be set and the `timeChanged` will be updated. If there is more data to write, the next `cow_block_metadata_t` will be computed and the steps above will be repeated.
The `workCounter` variable is used here again to make sure that if padding was needed it is done before the fuse request returns.
### Block Upload
-For block upload there is an background threads which loop periodically over all cow blocks and checks if `timeChanged` is not 0 and the time difference between now an `timeChanged` is larger than `COW_MIN_UPLOAD_DELAY`. If so, the block will be uploaded. The `timeChanged` before the upload will be temporary stored. After the upload `timeChanged` will be set to 0 if it still has the same time than temporary stored (if not there was an modification while the upload and it needs to be uploaded again). Once the image is unmounted `COW_MIN_UPLOAD_DELAY` is ignored an all blocks if a time of not 0 will be uploaded. The upload is done via an [rest request](#/api/file/update). There are two different limits for the number of parallel uploads in the [config.h](#config-variables).
+For block upload there is a background thread which loops periodically over all cow blocks and checks if `timeChanged` is not 0 and the time difference between now an `timeChanged` is larger than `COW_MIN_UPLOAD_DELAY`. If so, the block will be uploaded. The `timeChanged` before the upload will be temporary stored. After the upload `timeChanged` will be set to 0 if it still has the same time than temporary stored (if not there was an modification while the upload and it needs to be uploaded again). Once the image is unmounted `COW_MIN_UPLOAD_DELAY` is ignored an all blocks if a time of not 0 will be uploaded. The upload is done via an [rest request](#/api/file/update). There are two different limits for the number of parallel uploads in the [config.h](#config-variables).
## Files
If a new CoW session is started, a new `meta`, `data` and if set so in the Command line arguments a `status.txt` file is created.
@@ -130,10 +129,10 @@ ulspeed=0.00
```
- The `uuid` is the session uuid, which the cow server uses to identify the session.
-- The `state` is `backgroundUpload` if the image is still mounted, an cow blocks are uploaded in the background.
-It is `uploading` if the image got dismounted and all not yet uploaded blocks are uploading.
+- The `state` is `backgroundUpload` if the image is still mounted and cow blocks are uploaded in the background.
+It is `uploading` if the image got dismounted and all not yet uploaded blocks are uploaded.
it is `done` if the image got dismounted and all blocks are uploaded.
-- `inQueue` are the cow blocks which are currently uploaded or waiting for an free slot.
+- `inQueue` are the cow blocks which are currently uploaded or waiting for a free slot.
- `modifiedBlocks` are cow block which have changes which are not uploaded to the server yet, because the changes are to recent.
- `totalBlocksUploaded` the total amount of cow blocks uploaded since the image was mounted.
- `activeUploads` is the number blocks that are currently uploaded.
@@ -168,7 +167,7 @@ typedef struct cowfile_metadata_header
```
After this header at byte 8192 starts the l1 and then the l2 data structure mentioned above.
### data
-The `data` files contains the magicValue and at the 40 * 8 * 4096 Offset(capacity of one cowfile_metadata_header_t) starts the first block data.
+The `data` files contain the magicValue and at the 40 * 8 * 4096 Offset(capacity of one cowfile_metadata_header_t) starts the first block data.
@@ -199,7 +198,7 @@ This extension uses a new lock ```cow.l2CreateLock```. It is used when a new L2
### Config Variables
-The follwoing configuration variables have been added to ```config.h```.
+The following configuration variables have been added to ```config.h```.
```c
//config.h
// +++++ COW +++++
@@ -221,14 +220,14 @@ The follwoing configuration variables have been added to ```config.h```.
- ```COW_MIN_UPLOAD_DELAY``` defines the minimum time in seconds that must have elapsed since the last modification of a cow block before it is uploaded. This value can be fine tuned. A larger value usually reduces duplicate block uploads. While a lower value usually reduces the time for the final upload after the image got unmounted. If you define `COW_DUMP_BLOCK_UPLOADS` and have set the command line parameter `--cowStatFile`, then after the block upload is complete, a list of all blocks and sorted by the number of uploads will be dumped into status.txt. This can help adjusting `COW_MIN_UPLOAD_DELAY`.
-- ```COW_STATS_UPDATE_TIME``` defines the update frequency in seconds of the stdout print/ stats file update. Setting this to low could impact the performance since it hast to loop over all blocks.
-- ```COW_MAX_PARALLEL_UPLOADS``` defines to maximal number of parallel block uploads. These number is used once the image hast been dismounted and the final blocks are uploaded
+- ```COW_STATS_UPDATE_TIME``` defines the update frequency in seconds of the stdout print/ stats file update. Setting this too low could impact the performance since it hast to loop over all blocks.
+- ```COW_MAX_PARALLEL_UPLOADS``` defines to maximal number of parallel block uploads. These number is used once the image hast was dismounted and the final blocks are uploaded.
- ```COW_MAX_PARALLEL_BACKGROUND_UPLOADS``` defines to maximal number of parallel block uploads. These number is used will the image is still mounted and the user is still using it.
# REST Api
-To transfer the data to the cow server, the following rest api is used:
+To transfer the data to the cow server, the following rest API is used:
### /api/File/Create
@@ -259,7 +258,7 @@ This request is used once a new cow session is created. The returned guid is us
| ---- | ----------- |
| 200 | Success |
-Used for uploading a block of data. The blocknumber is the absolute block number. The body contains an "application/octet-stream" where the first bytes are the bitfield directly followed by the actual blockdata.
+Used for uploading a block of data. The blocknumber is the absolute block number. The body contains an "application/octet-stream" where the first bytes are the bit field directly followed by the actual blockdata.
### /api/File/StartMerge
@@ -287,7 +286,7 @@ Used to start the merging on the server.
| ---- | ----------- |
| 200 | Success |
-This request returns a list that contains Block Ids and the amount of times this block got uploaded, sorted by the amount of uploads. This is useful to adjusting the `COW_MIN_UPLOAD_DELAY`.
+This request returns a list that contains Block Ids and the amount of times this block got uploaded, sorted by the amount of uploads. This is useful to adjust the `COW_MIN_UPLOAD_DELAY`.
### /api/File/Status