Welcome to the forum, Guest

TOPIC: Import speed

Import speed 3 years 5 months ago #5498

itotm

Offline

Fresh Boarder

Posts: 2

Karma: 0

Hi, is there any option to optimize the speed of the import from the library? When I copy files from the server via SMB or WebDAV I get 30-40 MB/s, while inside YACReader the maximum speed is about 3 MB/s.
The administrator has disabled public write access.

Import speed 3 years 5 months ago #5499

Luis Ángel

Offline

Administrator

Posts: 2616

Thank you received: 543

Karma: 22

I have to look into this, it shouldn't be that slow, but it really depends on how fast the server is able to process the files. What are the specs of the machine you are running YACReaderLibrary in?
Contribute to the project becoming a patron: www.patreon.com/yacreader
You can also donate via Pay-Pal: www.paypal.com/donate?business=5TAMNQCDD...e=Support+YACReader\
The administrator has disabled public write access.

Import speed 3 years 5 months ago #5500

itotm

Offline

Fresh Boarder

Posts: 2

Karma: 0

The server is a virtual machine with Win10, 6GB of RAM and 4 cores CPU (i7 3770, the vm has access to all the cores) and the client is an iPad Air. I did some other test:
- using the import function from the YACReder library, the max speed I can reach is 6.5 MB/s
- using the import function, but with the Files option and an app that uses SMB (FileExplorer), I reach 13 MB/s
- using other methods to copy files (SMB or WebDAV directly, with FileExploer and other apps), I reach 16 MB/s
- I reached 40 MB/s only positioning the iPad near the wifi router, so I can say that normally the maximum speed is around 15-16 MB/s.
All was done from the same server.
So, the difference is not that big (sorry), but I think that some improvement is possible (I'm a developer so I always think that an improvement is possible :-), unfortunately I use C#, I looked at your repo but I don't have tools to use it).
Thank you.
The administrator has disabled public write access.

Import speed 3 years 5 months ago #5501

selmf

Offline

Developer

Posts: 631

Thank you received: 135

Karma: 18

I'm not quite up to speed with the current state of the server code, but from what I can remember a speedup likely could be had by adding more parallelism for file formats that are suited for it and smarter parsing/decompression of files that are not. With the current implementation and the intrinsic unsuitedness of many archive formats to parallel processing the speed of the server is limited by the single core speed of the cpu.
My answers are not necessarily official YACReader statements but mostly represent my own opinion in technical matters.
The administrator has disabled public write access.
Powered by Kunena Forum