[CODE] ZMemory - Dynamic Memory Allocation

Looking for Resources for your mod/project? Want to share some of your creations so others can use them? Post them here!
Post Reply
Lollipop
Zandrone
Posts: 1123
Joined: Tue Jul 24, 2012 10:34 am
Location: Denmark

[CODE] ZMemory - Dynamic Memory Allocation

#1

Post by Lollipop » Fri Aug 17, 2018 1:32 pm

Bob's button: Releases

ZMemory
This thing allocates memory dynamically in ACS. Malloc and free as you might know from C or similar.
It allows you to build data structures on the fly for things like dialogue systems, menu systems or whatever else. This can be used by mods to provide an API for addons to modify these structures without having to copy-paste your script and change what it needs to be different, allowing for a horizontal addon structure instead of imposing a strictly linear one. That is, different addons modifying your ACS data can do so simultaneously as long as they don't interact with conflicting semantics, but that's a semantics issue of the memory usage, which lies outside of the scope of this project to handle.

How do I do this? Global and world scope arrays can both be addressed absolutely ridiculously. I originally tested it with index twenty million and it worked. It even accepts negative indices, which means that the addressable size of these arrays is 2^32, which should be sufficient for all intents and purposes. This is clearly a hack, but one that works beautifully.

There is additional information in the readme.


The current version has major version 0, so expect the API to possibly change.
Spoiler: current API (Open)
  • gmalloc(int size)
  • gfree(int p_ptr)
  • wmalloc(int size)
  • wfree(int p_ptr)
Spoiler: Compatibility notice (Open)
This should compatible with anything that doesn't tamper with global array 63 or world array 255.
Final words: I am fully aware that this is not a tool with an abundance of use cases, however it is useful in certain scenarios and makes things possible that were not before.

Trivia: Project started on 11 august 2018

Download it here on the release page: Releases
Last edited by Lollipop on Sun Feb 02, 2020 11:54 am, edited 2 times in total.

User avatar
Ivan
Addicted to Zandronum
Posts: 2219
Joined: Mon Jun 04, 2012 5:38 pm
Location: Omnipresent

Re: [SCRIPT] ZMemory - Dynamic Memory Allocation

#2

Post by Ivan » Sat Aug 18, 2018 3:24 pm

I didn't know that global variables had a large index size. That's a good find. You might want to have the "log" calls disabled for the release by the way.
=== RAGNAROK DM ON ... uh... dead forever? ===
=== ALWAYS BET ON ... uh... dead forever? ===
=== Who wanta sum wang? ===
=== Death and Decay - A new Monster/Weapon replacer ===

Lollipop
Zandrone
Posts: 1123
Joined: Tue Jul 24, 2012 10:34 am
Location: Denmark

Re: [SCRIPT] ZMemory - Dynamic Memory Allocation

#3

Post by Lollipop » Sat Aug 18, 2018 3:55 pm

That is a good point. It just so happens that I'm an enormous derp.
I will fix that soon. I will also try to make the first stable release as soon as possible so that it can be actively used in projects.

User avatar
jdagenet
Forum Regular
Posts: 191
Joined: Tue Jun 05, 2012 8:08 am
Clan: Cube
Clan Tag: A3
Contact:

Re: [SCRIPT] ZMemory - Dynamic Memory Allocation

#4

Post by jdagenet » Sat Aug 18, 2018 5:59 pm

This seems really useful and could save a ton of work in the long run. Nice work.
<Dynamo_>uh
<Dynamo_>did you just take the thread away
<FusedQyou>Dunno
<FusedQyou>ask the thread

Lollipop
Zandrone
Posts: 1123
Joined: Tue Jul 24, 2012 10:34 am
Location: Denmark

Re: [SCRIPT] ZMemory - Dynamic Memory Allocation

#5

Post by Lollipop » Sat Aug 18, 2018 6:38 pm

Thank you. I will be very preoccupied until next weekend, but I will make the best of efforts to get the first stable version done then, at which point the API will be stable for that major release and it should be good to use.

Samuzero15tlh
Forum Regular
Posts: 253
Joined: Wed Sep 09, 2015 2:21 pm
Location: In home, sweet Home
Clan Tag: <skr>

Re: [CODE] ZMemory - Dynamic Memory Allocation

#6

Post by Samuzero15tlh » Mon Oct 14, 2019 3:28 am

Oh look!, a basic function to make dynamic lists! :D Now i need to remember how to use them on pointers...._.

Still this pice of code will help me out on the coding, thanks man!

User avatar
TDRR
Forum Regular
Posts: 214
Joined: Thu Jun 28, 2018 9:13 pm
Location: Venezuela

Re: [CODE] ZMemory - Dynamic Memory Allocation

#7

Post by TDRR » Mon Oct 14, 2019 3:14 pm

Global arrays (if they are set to the largest int, which they probably are since there's no other one-size-fits-all value) can hold 2 billion values!

Also, wouldn't it be possible to use a second global array to double the size? Once the end of the first one is reached, allocate memory on the second one. Or use all 64 global arrays to get 137438953408 values (one hundred and thirty seven billion, four hundred and thirty eight million, nine hundred and fifty three thousand, four hundred and eight) which is absolutely balls but could be really cool!
After a google search, i found out that a single array full of 0 or 1 values would be equal to... 265.4MB, and all 64 global arrays would be around 16GB O_O
That's a truckload of space for a scripting language like ACS!

What happened to this though, did it just stop or is it still in very slow development?
"I will find joy in Yahweh. I will delight in my Elohim. He has dressed me in the clothes of salvation. He has wrapped me in the robe of righteousness like a bridegroom with a priest’s turban, like a bride with her jewels."
-Isaiah 61:10 (Names of God Bible)

Lollipop
Zandrone
Posts: 1123
Joined: Tue Jul 24, 2012 10:34 am
Location: Denmark

Re: [CODE] ZMemory - Dynamic Memory Allocation

#8

Post by Lollipop » Sun Feb 02, 2020 12:02 pm

I have been quite busy for a long time, so it was kinda in development hell.

I have finally gotten around to building a metatool so I have one codebase for both world and global scope memory management, which is great, as I didn't want to bother with copy-pasting code and doing a bunch of edits all the time.
So that means v0.1.0 is out with the interface changing from malloc/free to gmalloc/gfree and the new wmalloc/wfree, for global and world scopes. The efficiency of the implementation hasn't changed in the slightest and it looks kinda messy with the way I got the metatool to work, but it should all work. Gotta admit that I haven't really tested it too much.

Regarding your calculations, we actually can use negative indices for some reason, giving 2^32 indices to work with, so unless my code contains an implementation detail that limits the memory space (I didn't check) then we should have 2^32 addresses, with is 4*2^30, each being 4 bytes gives 16 GB of addressable memory in each scope. I think this should be sufficient for most people, especially as there are many people who play with less than that in their machine.
If my further calculations are correct, then ACS can address 5TB of memory, which is ludicrous.

Thanks for the interaction, though. I didn't really think people would care after all this time, or much at all to be honest.

User avatar
TDRR
Forum Regular
Posts: 214
Joined: Thu Jun 28, 2018 9:13 pm
Location: Venezuela

Re: [CODE] ZMemory - Dynamic Memory Allocation

#9

Post by TDRR » Sat Feb 15, 2020 7:54 pm

Hey dude, i would recommend having a version of the ACS library that does not require using a build of ZMemory, because it is super inconvenient for the users to load another PK3 who probably have no idea what does it do or where to download it.

Not only that, but if i wanted to have a ZMemory fork with edits, bugfixes or any extra features, i'd have to figure out how the heck does that metatool business work and try to get something that works out of it, and manually, because for those dudes not in Linux or people who haven't set up a build environment, there's no other way out other than set up one or figure out how to turn the source into something ACC could actually compile.

I say this because really, i would like to use this but at the moment it's not worth the setup.
"I will find joy in Yahweh. I will delight in my Elohim. He has dressed me in the clothes of salvation. He has wrapped me in the robe of righteousness like a bridegroom with a priest’s turban, like a bride with her jewels."
-Isaiah 61:10 (Names of God Bible)

Lollipop
Zandrone
Posts: 1123
Joined: Tue Jul 24, 2012 10:34 am
Location: Denmark

Re: [CODE] ZMemory - Dynamic Memory Allocation

#10

Post by Lollipop » Wed Feb 19, 2020 3:34 pm

Well, you can simply take whichever build works and put it into your pk3 if you think your players will have problems. Besides, most of the time people play the kinds of mods for which this would potentially be useful on servers that have figured out the load order business anyways. It isn't a problem without solutions. People are not stupid either.

About the metatool, all I can really say is tough shit. It makes my life easier and it was never intended to solve any other problem than me not having to copy paste things, potentially introducing mistakes in that process.
I could potentially document the metatool, but it's such a hackjob that it would just mislead people into thinking that it's something valuable. Besides, the way it works now is a no-brainer, aside from a couple of quirks that is.
Regarding linuxness, that is what I use. I don't use windows, mac or anything else at the moment so I can't even make a build environment for those systems as I don't own them. I also don't particularly care. If someone actually wants to mess with memory allocators then I expect that they are at a level where these things are not a problem at all. If it is, then don't mess with memory allocators. It isn't an easy topic, even if the one I have built as a proof of concept is fairly simple in its construction.

Also, what setup? You don't need to build it yourself. Just grab the allocator pk3 and the ZMemory.acs interface file and work away. The interface file is all you need to include from your own ACS and just load the allocator first in the load order. That's all.
Don't mess around in the repo for what you need, it's all in the release page.

Thanks for showing interest, though. (^:

Post Reply