summaryrefslogtreecommitdiffstats
path: root/ui/vnc-enc-zrle.c
Commit message (Collapse)AuthorAgeFilesLines
* meson: rename included C source files to .c.incPaolo Bonzini2020-08-211-11/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | With Makefiles that have automatically generated dependencies, you generated includes are set as dependencies of the Makefile, so that they are built before everything else and they are available when first building the .c files. Alternatively you can use a fine-grained dependency, e.g. target/arm/translate.o: target/arm/decode-neon-shared.inc.c With Meson you have only one choice and it is a third option, namely "build at the beginning of the corresponding target"; the way you express it is to list the includes in the sources of that target. The problem is that Meson decides if something is a source vs. a generated include by looking at the extension: '.c', '.cc', '.m', '.C' are sources, while everything else is considered an include---including '.inc.c'. Use '.c.inc' to avoid this, as it is consistent with our other convention of using '.rst.inc' for included reStructuredText files. The editorconfig file is adjusted. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* vnc: prioritize ZRLE compression over ZLIBCameron Esfahani2020-01-211-2/+2
| | | | | | | | | | | | | In my investigation, ZRLE always compresses better than ZLIB so prioritize ZRLE over ZLIB, even if the client hints that ZLIB is preferred. zlib buffer is always reset in zrle_compress_data(), so using offset to calculate next_out and avail_out is useless. Signed-off-by: Cameron Esfahani <dirty@apple.com> Message-Id: <b5d129895d08a90d0a2a6183b95875bacfa998b8.1579582674.git.dirty@apple.com> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
* vnc: fix memory leak when vnc disconnectLi Qiang2019-09-171-34/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently when qemu receives a vnc connect, it creates a 'VncState' to represent this connection. In 'vnc_worker_thread_loop' it creates a local 'VncState'. The connection 'VcnState' and local 'VncState' exchange data in 'vnc_async_encoding_start' and 'vnc_async_encoding_end'. In 'zrle_compress_data' it calls 'deflateInit2' to allocate the libz library opaque data. The 'VncState' used in 'zrle_compress_data' is the local 'VncState'. In 'vnc_zrle_clear' it calls 'deflateEnd' to free the libz library opaque data. The 'VncState' used in 'vnc_zrle_clear' is the connection 'VncState'. In currently implementation there will be a memory leak when the vnc disconnect. Following is the asan output backtrack: Direct leak of 29760 byte(s) in 5 object(s) allocated from: 0 0xffffa67ef3c3 in __interceptor_calloc (/lib64/libasan.so.4+0xd33c3) 1 0xffffa65071cb in g_malloc0 (/lib64/libglib-2.0.so.0+0x571cb) 2 0xffffa5e968f7 in deflateInit2_ (/lib64/libz.so.1+0x78f7) 3 0xaaaacec58613 in zrle_compress_data ui/vnc-enc-zrle.c:87 4 0xaaaacec58613 in zrle_send_framebuffer_update ui/vnc-enc-zrle.c:344 5 0xaaaacec34e77 in vnc_send_framebuffer_update ui/vnc.c:919 6 0xaaaacec5e023 in vnc_worker_thread_loop ui/vnc-jobs.c:271 7 0xaaaacec5e5e7 in vnc_worker_thread ui/vnc-jobs.c:340 8 0xaaaacee4d3c3 in qemu_thread_start util/qemu-thread-posix.c:502 9 0xffffa544e8bb in start_thread (/lib64/libpthread.so.0+0x78bb) 10 0xffffa53965cb in thread_start (/lib64/libc.so.6+0xd55cb) This is because the opaque allocated in 'deflateInit2' is not freed in 'deflateEnd'. The reason is that the 'deflateEnd' calls 'deflateStateCheck' and in the latter will check whether 's->strm != strm'(libz's data structure). This check will be true so in 'deflateEnd' it just return 'Z_STREAM_ERROR' and not free the data allocated in 'deflateInit2'. The reason this happens is that the 'VncState' contains the whole 'VncZrle', so when calling 'deflateInit2', the 's->strm' will be the local address. So 's->strm != strm' will be true. To fix this issue, we need to make 'zrle' of 'VncState' to be a pointer. Then the connection 'VncState' and local 'VncState' exchange mechanism will work as expection. The 'tight' of 'VncState' has the same issue, let's also turn it to a pointer. Reported-by: Ying Fang <fangying1@huawei.com> Signed-off-by: Li Qiang <liq3ea@163.com> Message-id: 20190831153922.121308-1-liq3ea@163.com Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
* rename included C files to foo.inc.c, remove osdep.hPaolo Bonzini2018-05-111-11/+11
| | | | | | | | | | | osdep.h is only needed for files that are compiled directly. Remove it from included C source files, and rename them to *.inc.c so that scripts/clean-includes knows to skip them. Cc: Eric Blake <eblake@redhat.com> Cc: Michael S. Tsirkin <mst@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* vnc: simple clean upWei Qi2017-05-121-1/+0Star
| | | | | | | | It is unnecessary to assign 'packed_bytes' to 'estimated_bytes', because 'estimated_bytes' unused after assignment. Signed-off-by: Wei Qi <weiqi4@huawei.com> Reviewed-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@redhat.com> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
* ui: Clean up includesPeter Maydell2016-02-041-0/+1
| | | | | | | | | | Clean up includes so that osdep.h is included first and headers which it implies are not included manually. This commit was created with scripts/clean-includes. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 1454089805-5470-2-git-send-email-peter.maydell@linaro.org
* pixman/vnc: use pixman images in vnc.Gerd Hoffmann2012-11-011-9/+9
| | | | | | | | | | | | | | | | | | | | | | | The vnc code uses *three* DisplaySurfaces: First is the surface of the actual QemuConsole, usually the guest screen, but could also be a text console (monitor/serial reachable via Ctrl-Alt-<nr> keys). This is left as-is. Second is the current server's view of the screen content. The vnc code uses this to figure which parts of the guest screen did _really_ change to reduce the amount of updates sent to the vnc clients. It is also used as data source when sending out the updates to the clients. This surface gets replaced by a pixman image. The format changes too, instead of using the guest screen format we'll use fixed 32bit rgb framebuffer and convert the pixels on the fly when comparing and updating the server framebuffer. Third surface carries the format expected by the vnc client. That isn't used to store image data. This surface is switched to PixelFormat and a boolean for bigendian byte order. Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
* vnc: fix uint8_t comparisons with negative valuesCorentin Chary2011-02-231-1/+2
| | | | | Signed-off-by: Corentin Chary <corentincj@iksaif.net> Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
* vnc: Add ZRLE and ZYWRLE encodings.Corentin Chary2011-02-231-0/+365
Add ZRLE [1] and ZYWRLE [2] encodings. The code is inspire^W stolen from libvncserver (again), but have been rewriten to match QEMU coding style. [1] http://www.realvnc.com/docs/rfbproto.pdf [2] http://micro-vnc.jp/research/remote_desktop_ng/ZYWRLE/publications/ Signed-off-by: Corentin Chary <corentincj@iksaif.net> Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>