summaryrefslogtreecommitdiffstats
path: root/block/file-posix.c
Commit message (Collapse)AuthorAgeFilesLines
* file-posix: Do runtime check for ofd lock APIFam Zheng2017-08-111-11/+8Star
| | | | | | | | | | | | | | | | | | | | | | | | It is reported that on Windows Subsystem for Linux, ofd operations fail with -EINVAL. In other words, QEMU binary built with system headers that exports F_OFD_SETLK doesn't necessarily run in an environment that actually supports it: $ qemu-system-aarch64 ... -drive file=test.vhdx,if=none,id=hd0 \ -device virtio-blk-pci,drive=hd0 qemu-system-aarch64: -drive file=test.vhdx,if=none,id=hd0: Failed to unlock byte 100 qemu-system-aarch64: -drive file=test.vhdx,if=none,id=hd0: Failed to unlock byte 100 qemu-system-aarch64: -drive file=test.vhdx,if=none,id=hd0: Failed to lock byte 100 As a matter of fact this is not WSL specific. It can happen when running a QEMU compiled against a newer glibc on an older kernel, such as in a containerized environment. Let's do a runtime check to cope with that. Reported-by: Andrew Baumann <Andrew.Baumann@microsoft.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Fam Zheng <famz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* block: respect error code from bdrv_getlength in handle_aiocb_write_zeroesDenis V. Lunev2017-08-081-1/+7
| | | | | | | | | | | | | | | | Original idea beyond the code in question was the following: we have failed to write zeroes with fallocate(FALLOC_FL_ZERO_RANGE) as the simplest approach and via fallocate(FALLOC_FL_PUNCH_HOLE)/fallocate(0). We have the only chance now: if the request comes beyond end of the file. Thus we should calculate file length and respect the error code from that op. Signed-off-by: Denis V. Lunev <den@openvz.org> CC: Markus Armbruster <armbru@redhat.com> CC: Kevin Wolf <kwolf@redhat.com> CC: Max Reitz <mreitz@redhat.com> CC: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
* block/file-posix: Preallocation for truncateMax Reitz2017-07-111-12/+10Star
| | | | | | | | | | By using raw_regular_truncate() in raw_truncate(), we can now easily support preallocation. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20170613202107.10125-9-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
* block/file-posix: Generalize raw_regular_truncateMax Reitz2017-07-111-11/+50
| | | | | | | | | | | | | | | | Currently, raw_regular_truncate() is intended for setting the size of a newly created file. However, we also want to use it for truncating an existing file in which case only the newly added space (when growing) should be preallocated. This also means that if resizing failed, we should try to restore the original file size. This is important when using preallocation. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20170613202107.10125-8-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
* block/file-posix: Extract raw_regular_truncate()Max Reitz2017-07-111-66/+78
| | | | | | | | | | | This functionality is part of raw_create() which we will be able to reuse nicely in raw_truncate(). Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20170613202107.10125-7-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
* block/file-posix: Small fixes in raw_create()Max Reitz2017-07-111-2/+3
| | | | | | | | | | | | Variables should be declared at the start of a block, and if a certain parameter value is not supported it may be better to return -ENOTSUP instead of -EINVAL. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20170613202107.10125-6-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
* block: Add PreallocMode to BD.bdrv_truncate()Max Reitz2017-07-111-1/+8
| | | | | | | | | | | Add a PreallocMode parameter to the bdrv_truncate() function implemented by each block driver. Currently, we always pass PREALLOC_MODE_OFF and no driver accepts anything else. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20170613202107.10125-2-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
* block: change variable names in BlockDriverStateManos Pitsidianakis2017-06-261-17/+17
| | | | | | | | | | | Change the 'int count' parameter in *pwrite_zeros, *pdiscard related functions (and some others) to 'int bytes', as they both refer to bytes. This helps with code legibility. Signed-off-by: Manos Pitsidianakis <el13635@mail.ntua.gr> Message-id: 20170609101808.13506-1-el13635@mail.ntua.gr Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
* block/file-*: *_parse_filename() and colonsMax Reitz2017-05-291-14/+3Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The file drivers' *_parse_filename() implementations just strip the optional protocol prefix off the filename. However, for e.g. "file:foo:bar", this would lead to "foo:bar" being stored as the BDS's filename which looks like it should be managed using the "foo" protocol. This is especially troublesome if you then try to resolve a backing filename based on "foo:bar". This issue can only occur if the stripped part is a relative filename ("file:/foo:bar" will be shortened to "/foo:bar" and having a slash before the first colon means that "/foo" is not recognized as a protocol part). Therefore, we can easily fix it by prepending "./" to such filenames. Before this patch: $ ./qemu-img create -f qcow2 backing.qcow2 64M Formatting 'backing.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 $ ./qemu-img create -f qcow2 -b backing.qcow2 file:top:image.qcow2 Formatting 'file:top:image.qcow2', fmt=qcow2 size=67108864 backing_file=backing.qcow2 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 $ ./qemu-io file:top:image.qcow2 can't open device file:top:image.qcow2: Could not open backing file: Unknown protocol 'top' After this patch: $ ./qemu-io file:top:image.qcow2 [no error] Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20170522195217.12991-3-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
* file-posix: Remove .bdrv_inactivate/invalidate_cacheKevin Wolf2017-05-111-33/+0Star
| | | | | | | | | Now that the block layer takes care to request a lot less permissions for inactive nodes, the special-casing in file-posix isn't necessary any more. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
* file-posix: Add image locking to perm operationsFam Zheng2017-05-111-1/+275
| | | | | | | | | | | | | | | | | | This extends the permission bits of op blocker API to external using Linux OFD locks. Each permission in @perm and @shared_perm is represented by a locked byte in the image file. Requesting a permission in @perm is translated to a shared lock of the corresponding byte; rejecting to share the same permission is translated to a shared lock of a separate byte. With that, we use 2x number of bytes of distinct permission types. virtlockd in libvirt locks the first byte, so we do locking from a higher offset. Suggested-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Fam Zheng <famz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* file-posix: Add 'locking' optionFam Zheng2017-05-111-0/+5
| | | | | | | | | | Making this option available even before implementing it will let converting tests easier: in coming patches they can specify the option already when necessary, before we actually write code to lock the images. Signed-off-by: Fam Zheng <famz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* qobject: Use simpler QDict/QList scalar insertion macrosEric Blake2017-05-091-4/+4
| | | | | | | | | | | | | | | | | | We now have macros in place to make it less verbose to add a scalar to QDict and QList, so use them. Patch created mechanically via: spatch --sp-file scripts/coccinelle/qobject.cocci \ --macro-file scripts/cocci-macro-file.h --dir . --in-place then touched up manually to fix a couple of '?:' back to original spacing, as well as avoiding a long line in monitor.c. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20170427215821.19397-7-eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qobject: Drop useless QObject castsEric Blake2017-05-081-3/+3
| | | | | | | | | | | | | | | | | | | We have macros in place to make it less verbose to add a subtype of QObject to both QDict and QList. While we have made cleanups like this in the past (see commit fcfcd8ffc, for example), having it be automated by Coccinelle makes it easier to maintain. Patch created mechanically via: spatch --sp-file scripts/coccinelle/qobject.cocci \ --macro-file scripts/cocci-macro-file.h --dir . --in-place then I verified that no manual touchups were required. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20170427215821.19397-5-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* block: Add .bdrv_truncate() error messagesMax Reitz2017-04-281-5/+12
| | | | | | | | | | | | | | Add missing error messages for the block driver implementations of .bdrv_truncate(); drop the generic one from block.c's bdrv_truncate(). Since one of these changes touches a mis-indented block in block/file-posix.c, this patch fixes that coding style issue along the way. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20170328205129.15138-5-mreitz@redhat.com Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
* block: Add errp to BD.bdrv_truncate()Max Reitz2017-04-281-1/+1
| | | | | | | | | | | | | | Add an Error parameter to the block drivers' bdrv_truncate() interface. If a block driver does not set this in case of an error, the generic bdrv_truncate() implementation will do so. Where it is obvious, this patch also makes some block drivers set this value. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20170328205129.15138-4-mreitz@redhat.com Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
* file-posix: Remove unnecessary includesKevin Wolf2017-04-271-2/+0Star
| | | | Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* block: Document -drive problematic code and bugsMarkus Armbruster2017-04-031-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | -blockdev and blockdev_add convert their arguments via QObject to BlockdevOptions for qmp_blockdev_add(), which converts them back to QObject, then to a flattened QDict. The QDict's members are typed according to the QAPI schema. -drive converts its argument via QemuOpts to a (flat) QDict. This QDict's members are all QString. Thus, the QType of a flat QDict member depends on whether it comes from -drive or -blockdev/blockdev_add, except when the QAPI type maps to QString, which is the case for 'str' and enumeration types. The block layer core extracts generic configuration from the flat QDict, and the block driver extracts driver-specific configuration. Both commonly do so by converting (parts of) the flat QDict to QemuOpts, which turns all values into strings. Not exactly elegant, but correct. However, A few places access the flat QDict directly: * Most of them access members that are always QString. Correct. * bdrv_open_inherit() accesses a boolean, carefully. Correct. * nfs_config() uses a QObject input visitor. Correct only because the visited type contains nothing but QStrings. * nbd_config() and ssh_config() use a QObject input visitor, and the visited types contain non-QStrings: InetSocketAddress members @numeric, @to, @ipv4, @ipv6. -drive works as long as you don't try to use them (they're all optional). @to is ignored anyway. Reproducer: -drive driver=ssh,server.host=h,server.port=22,server.ipv4,path=p -drive driver=nbd,server.type=inet,server.data.host=h,server.data.port=22,server.data.ipv4 both fail with "Invalid parameter type for 'data.ipv4', expected: boolean" Add suitable comments to all these places. Mark the buggy ones FIXME. "Fortunately", -drive's driver-specific options are entirely undocumented. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-id: 1490895797-29094-5-git-send-email-armbru@redhat.com [mreitz: Fixed two typos] Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
* block/file-posix.c: Fix unused variable warning on OpenBSDPeter Maydell2017-03-271-14/+14
| | | | | | | | | | | | On OpenBSD none of the ioctls probe_logical_blocksize() tries exist, so the variable sector_size is unused. Refactor the code to avoid this (and reduce the duplicated code). Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Jeff Cody <jcody@redhat.com> Message-id: 1490279788-12995-1-git-send-email-peter.maydell@linaro.org Signed-off-by: Max Reitz <mreitz@redhat.com>
* file-posix: Make bdrv_flush() failure permanent without O_DIRECTKevin Wolf2017-03-271-0/+22
| | | | | | | | | | | | | | | | | Success for bdrv_flush() means that all previously written data is safe on disk. For fdatasync(), the best semantics we can hope for on Linux (without O_DIRECT) is that all data that was written since the last call was successfully written back. Therefore, and because we can't redo all writes after a flush failure, we have to give up after a single fdatasync() failure. After this failure, we would never be able to make the promise that a successful bdrv_flush() makes. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20170322210005.16533-1-kwolf@redhat.com Reviewed-by: Fam Zheng <famz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
* file-posix: Don't leak fd in hdev_get_max_segmentsFam Zheng2017-03-171-0/+3
| | | | | | | | This fixes a leaked fd introduced in commit 9103f1ce. Signed-off-by: Fam Zheng <famz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* file-posix: clean up max_segments buffer terminationStefan Hajnoczi2017-03-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | | The following pattern is unsafe: char buf[32]; ret = read(fd, buf, sizeof(buf)); ... buf[ret] = 0; If read(2) returns 32 then a byte beyond the end of the buffer is zeroed. In practice this buffer overflow does not occur because the sysfs max_segments file only contains an unsigned short + '\n'. The string is always shorter than 32 bytes. Regardless, avoid this pattern because static analysis tools might complain and it could lead to real buffer overflows if copy-pasted elsewhere in the codebase. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* file-posix: Consider max_segments for BlockLimits.max_transferFam Zheng2017-03-131-0/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | BlockLimits.max_transfer can be too high without this fix, guest will encounter I/O error or even get paused with werror=stop or rerror=stop. The cause is explained below. Linux has a separate limit, /sys/block/.../queue/max_segments, which in the worst case can be more restrictive than the BLKSECTGET which we already consider (note that they are two different things). So, the failure scenario before this patch is: 1) host device has max_sectors_kb = 4096 and max_segments = 64; 2) guest learns max_sectors_kb limit from QEMU, but doesn't know max_segments; 3) guest issues e.g. a 512KB request thinking it's okay, but actually it's not, because it will be passed through to host device as an SG_IO req that has niov > 64; 4) host kernel doesn't like the segmenting of the request, and returns -EINVAL; This patch checks the max_segments sysfs entry for the host device and calculates a "conservative" bytes limit using the page size, which is then merged into the existing max_transfer limit. Guest will discover this from the usual virtual block device interfaces. (In the case of scsi-generic, it will be done in the INQUIRY reply interception in device model.) The other possibility is to actually propagate it as a separate limit, but it's not better. On the one hand, there is a big complication: the limit is per-LUN in QEMU PoV (because we can attach LUNs from different host HBAs to the same virtio-scsi bus), but the channel to communicate it in a per-LUN manner is missing down the stack; on the other hand, two limits versus one doesn't change much about the valid size of I/O (because guest has no control over host segmenting). Also, the idea to fall back to bounce buffering in QEMU, upon -EINVAL, was explored. Unfortunately there is no neat way to ensure the bounce buffer is less segmented (in terms of DMA addr) than the guest buffer. Practically, this bug is not very common. It is only reported on a Emulex (lpfc), so it's okay to get it fixed in the easier way. Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Fam Zheng <famz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* qemu-img: Improve documentation for PREALLOC_MODE_FALLOCNir Soffer2017-02-241-1/+6
| | | | | | | | | Now that we are truncating the file in both PREALLOC_MODE_FULL and PREALLOC_MODE_OFF, not truncating in PREALLOC_MODE_FALLOC looks odd. Add a comment explaining why we do not truncate in this case. Signed-off-by: Nir Soffer <nirsof@gmail.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* qemu-img: Truncate before full preallocationNir Soffer2017-02-241-0/+12
| | | | | | | | | | | | | | | | In a previous commit (qemu-img: Do not truncate before preallocation) we moved truncate to the PREALLOC_MODE_OFF branch to avoid slowdown in posix_fallocate(). However this change is not optimal when using PREALLOC_MODE_FULL, since knowing the final size from the beginning could allow the file system driver to do less allocations and possibly avoid fragmentation of the file. Now we truncate also before doing full preallocation. Signed-off-by: Nir Soffer <nirsof@gmail.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* qemu-img: Do not truncate before preallocationNir Soffer2017-02-241-7/+4Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When using file system that does not support fallocate() (e.g. NFS < 4.2), truncating the file only when preallocation=OFF speeds up creating raw file. Here is example run, tested on Fedora 24 machine, creating raw file on NFS version 3 server. $ time ./qemu-img-master create -f raw -o preallocation=falloc mnt/test 1g Formatting 'mnt/test', fmt=raw size=1073741824 preallocation=falloc real 0m21.185s user 0m0.022s sys 0m0.574s $ time ./qemu-img-fix create -f raw -o preallocation=falloc mnt/test 1g Formatting 'mnt/test', fmt=raw size=1073741824 preallocation=falloc real 0m11.601s user 0m0.016s sys 0m0.525s $ time dd if=/dev/zero of=mnt/test bs=1M count=1024 oflag=direct 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 15.6627 s, 68.6 MB/s real 0m16.104s user 0m0.009s sys 0m0.220s Running with strace we can see that without this change we do one pread() and one pwrite() for each block. With this change, we do only one pwrite() per block. $ strace ./qemu-img-master create -f raw -o preallocation=falloc mnt/test 8192 ... pread64(9, "\0", 1, 4095) = 1 pwrite64(9, "\0", 1, 4095) = 1 pread64(9, "\0", 1, 8191) = 1 pwrite64(9, "\0", 1, 8191) = 1 $ strace ./qemu-img-fix create -f raw -o preallocation=falloc mnt/test 8192 ... pwrite64(9, "\0", 1, 4095) = 1 pwrite64(9, "\0", 1, 8191) = 1 This happens because posix_fallocate is checking if each block is allocated before writing a byte to the block, and when truncating the file before preallocation, all blocks are unallocated. Signed-off-by: Nir Soffer <nirsof@gmail.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* block: get max_transfer limit for char (scsi-generic) devicesEric Farman2017-01-271-1/+1
| | | | | | | | | | | | | We can get the maximum number of bytes for a single I/O transfer from the BLKSECTGET ioctl, but we only perform this for block devices. scsi-generic devices are represented as character devices, and so do not issue this today. Update this, so that virtio-scsi devices using the scsi-generic interface can return the same data. Signed-off-by: Eric Farman <farman@linux.vnet.ibm.com> Message-Id: <20170120162527.66075-4-farman@linux.vnet.ibm.com> Reviewed-by: Fam Zheng <famz@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* block: Fix target variable of BLKSECTGET ioctlEric Farman2017-01-271-7/+10
| | | | | | | | | | | | | | | | | | | | Commit 6f6071745bd0 ("raw-posix: Fetch max sectors for host block device") introduced a routine to call the kernel BLKSECTGET ioctl, which stores the result back to user space. However, the size of the data returned depends on the routine handling the ioctl. The (compat_)blkdev_ioctl returns a short, while sg_ioctl returns an int. Thus, on big-endian systems, we can find ourselves accidentally shifting the result to a much larger value. (On s390x, a short is 16 bits while an int is 32 bits.) Also, the two ioctl handlers return values in different scales (block returns sectors, while sg returns bytes), so some tweaking of the outputs is required such that hdev_get_max_transfer_length returns a value in a consistent set of units. Signed-off-by: Eric Farman <farman@linux.vnet.ibm.com> Message-Id: <20170120162527.66075-3-farman@linux.vnet.ibm.com> Reviewed-by: Fam Zheng <famz@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* block: Rename raw-{posix,win32} to file-*.cEric Blake2017-01-091-0/+2616
These files deal with the file protocol, not the raw format (the file protocol is often used with other formats, and the raw format is not forced to use the file protocol). Rename things to make it a bit easier to follow. Suggested-by: Daniel P. Berrange <berrange@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Laszlo Ersek <lersek@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
generated by cgit v1.2.3-55-g7522 (git 2.39.0) at 2024-12-22 13:06:18 +0100