Page MenuHomeFeedback Tracker

PBO with Compression by default
Closed, ResolvedPublic

Description

Since arma introduced the format PBO to pack missions and mods, the world evolved.
Files are now bigger, images etc are included inside pbos.

The problem with pbo is, there's no compression at all BY DEFAULT!
All files are concatenated and prefixed by a format identifier, that's all.

Why pbos don't have compression?
Because arma reads them for example while selecting a mission. It would be hard to decompress every pbo every time the game needs information about the content.

What's the idea of compressed pbos?
The idea is to partially compress pbos. There's only the description.ext needed to read without extracting. But inside the description.ext there are includes for example which will break a simple solution.

New Concept:
Introduce a simple metadata file (like description.ext) which has all information needed for the game when it's scanning the mission directory (gametype, respawn-type, mission name, mission description).
This file, i call it metadata, is available uncompressed at the beginning of each compressed-pbo (cpbo). All other data is compressed and appended after that.
That would lead to a big improvement of filesize (up to 60% less). But there may be a small problem you may have noticed: Files need to be extracted, in pbos they are read in memory at a time.
The solution is the deflate algorithm, if you need partial decompression: http://en.wikipedia.org/wiki/DEFLATE
It's used in many compressions today and it's really simple, because it's possible to create tables to map compressed data to uncompressed data, the file could be read and decompressed partially on-fly.

Details

Legacy ID
766184159
Severity
None
Resolution
Not A Bug
Reproducibility
N/A
Category
Engine
Additional Information

ZSync uses this theory of mapping deflated files and describes it in a tech paper: http://zsync.moria.org.uk/paper200503/ch03s02.html

Event Timeline

coati edited Steps To Reproduce. (Show Details)Jul 24 2013, 11:44 AM
coati edited Additional Information. (Show Details)
coati set Category to Engine.
coati set Reproducibility to N/A.
coati set Severity to None.
coati set Resolution to Not A Bug.
coati set Legacy ID to 766184159.May 7 2016, 3:39 PM

PBOs already support LZMA compression for whichever files you chose to apply it to, see Mikero's tools.

coati added a subscriber: coati.May 7 2016, 3:39 PM
coati added a comment.Jul 24 2013, 7:42 PM

oh ok, thank you very much, never heard of compression inside pbos :)
Close or should i change it to enable LZMA as a default?

Edit: Well thats a issue for modders not for bi, so close plz

Closed per request.