Possibilities for allocating memory for modular firmware design in C

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0;







up vote
16
down vote

favorite
9












modular approaches are pretty handy in general (portable and clean), so I try to program modules as independent of any other modules as possible. Most of my approaches are based on a struct that describes the module itself. An initialization function sets the primary parameters, afterwards a handler (pointer to desriptive struct) is passed to whatever function inside the module is called.



Right now, I am wondering what the best approach of allocation memory for the struct describing a module may be. If possible, I'd like the following:



  • Opaque struct, so the struct may only be altered by the use of provided interface functions

  • Multiple instances

  • memory allocated by linker

I see the following possibilities, that all conflict with one of my goals:



global declaration



multiple instances, allcoted by linker, but struct is not opaque



(#includes)
module_struct module;

void main()
module_init(&module);



malloc



opaque struct, multiple instances, but allcotion on heap



in module.h:



typedef module_struct Module;


in module.c init function, malloc and return pointer to allocated memory



module_mem = malloc(sizeof(module_struct ));
/* initialize values here */
return module_mem;


in main.c



(#includes)
Module *module;

void main()
module = module_init();



declaration in module



opaque struct, allocated by linker, only a predefined number of instances



keep the whole struct and memory internal to the module and never expose a handler or struct.



(#includes)

void main()
module_init(_no_param_or_index_if_multiple_instances_possible_);




Is there an option to combine these somehow for opaque struct, linker instead of heap allocation and multiple/any number of instances?



solution



as proposed in some answers below, I think the best way is to:



  1. reserve space for MODULE_MAX_INSTANCE_COUNT modules in the modules source file

  2. do not define MODULE_MAX_INSTANCE_COUNT in the module itsself

  3. add an #ifndef MODULE_MAX_INSTANCE_COUNT #error to the modules header file to make sure the modules user is aware of this limitation and defines the maximum number of instances wanted for the application

  4. on initialization of an instance return either the memory address (*void) of the desricptive struct or the modules index (whatever you like more)






share|improve this question


















  • 12




    Most embedded FW designers are avoiding the dynamic allocation to keep the memory usage deterministic and simple. Especially if it is bare-metal and does not have underlying OS to manage the memory.
    – Eugene Sh.
    Aug 15 at 14:25











  • Exactly, thats why I want the linker to do the allocations.
    – L. Heinrichs
    Aug 15 at 14:28






  • 4




    I'm not quite sure I understand... How could you have the memory allocated by the linker if you have a dynamic number of instances? That seems quite orthogonal to me.
    – jcaron
    Aug 15 at 17:13










  • Why not let the linker allocate one big memory pool, and do your own allocations from that, which also gives you the benefit of a zero-overhead allocator. You can make the pool object static to the file with the allocation function so it's private. In some of my code, I do all the allocations in the various init routines, then I print out afterward how much was allocated, so in the final production compile I set the pool to that exact size.
    – Lee Daniel Crocker
    Aug 15 at 23:58







  • 2




    If it's a compile-time decision, you could simply define the number in your Makefile or equivalent, and you're all set. The number would not be in the source of the module, but would be application specific, and you just use an instance number as a parameter.
    – jcaron
    Aug 16 at 14:49
















up vote
16
down vote

favorite
9












modular approaches are pretty handy in general (portable and clean), so I try to program modules as independent of any other modules as possible. Most of my approaches are based on a struct that describes the module itself. An initialization function sets the primary parameters, afterwards a handler (pointer to desriptive struct) is passed to whatever function inside the module is called.



Right now, I am wondering what the best approach of allocation memory for the struct describing a module may be. If possible, I'd like the following:



  • Opaque struct, so the struct may only be altered by the use of provided interface functions

  • Multiple instances

  • memory allocated by linker

I see the following possibilities, that all conflict with one of my goals:



global declaration



multiple instances, allcoted by linker, but struct is not opaque



(#includes)
module_struct module;

void main()
module_init(&module);



malloc



opaque struct, multiple instances, but allcotion on heap



in module.h:



typedef module_struct Module;


in module.c init function, malloc and return pointer to allocated memory



module_mem = malloc(sizeof(module_struct ));
/* initialize values here */
return module_mem;


in main.c



(#includes)
Module *module;

void main()
module = module_init();



declaration in module



opaque struct, allocated by linker, only a predefined number of instances



keep the whole struct and memory internal to the module and never expose a handler or struct.



(#includes)

void main()
module_init(_no_param_or_index_if_multiple_instances_possible_);




Is there an option to combine these somehow for opaque struct, linker instead of heap allocation and multiple/any number of instances?



solution



as proposed in some answers below, I think the best way is to:



  1. reserve space for MODULE_MAX_INSTANCE_COUNT modules in the modules source file

  2. do not define MODULE_MAX_INSTANCE_COUNT in the module itsself

  3. add an #ifndef MODULE_MAX_INSTANCE_COUNT #error to the modules header file to make sure the modules user is aware of this limitation and defines the maximum number of instances wanted for the application

  4. on initialization of an instance return either the memory address (*void) of the desricptive struct or the modules index (whatever you like more)






share|improve this question


















  • 12




    Most embedded FW designers are avoiding the dynamic allocation to keep the memory usage deterministic and simple. Especially if it is bare-metal and does not have underlying OS to manage the memory.
    – Eugene Sh.
    Aug 15 at 14:25











  • Exactly, thats why I want the linker to do the allocations.
    – L. Heinrichs
    Aug 15 at 14:28






  • 4




    I'm not quite sure I understand... How could you have the memory allocated by the linker if you have a dynamic number of instances? That seems quite orthogonal to me.
    – jcaron
    Aug 15 at 17:13










  • Why not let the linker allocate one big memory pool, and do your own allocations from that, which also gives you the benefit of a zero-overhead allocator. You can make the pool object static to the file with the allocation function so it's private. In some of my code, I do all the allocations in the various init routines, then I print out afterward how much was allocated, so in the final production compile I set the pool to that exact size.
    – Lee Daniel Crocker
    Aug 15 at 23:58







  • 2




    If it's a compile-time decision, you could simply define the number in your Makefile or equivalent, and you're all set. The number would not be in the source of the module, but would be application specific, and you just use an instance number as a parameter.
    – jcaron
    Aug 16 at 14:49












up vote
16
down vote

favorite
9









up vote
16
down vote

favorite
9






9





modular approaches are pretty handy in general (portable and clean), so I try to program modules as independent of any other modules as possible. Most of my approaches are based on a struct that describes the module itself. An initialization function sets the primary parameters, afterwards a handler (pointer to desriptive struct) is passed to whatever function inside the module is called.



Right now, I am wondering what the best approach of allocation memory for the struct describing a module may be. If possible, I'd like the following:



  • Opaque struct, so the struct may only be altered by the use of provided interface functions

  • Multiple instances

  • memory allocated by linker

I see the following possibilities, that all conflict with one of my goals:



global declaration



multiple instances, allcoted by linker, but struct is not opaque



(#includes)
module_struct module;

void main()
module_init(&module);



malloc



opaque struct, multiple instances, but allcotion on heap



in module.h:



typedef module_struct Module;


in module.c init function, malloc and return pointer to allocated memory



module_mem = malloc(sizeof(module_struct ));
/* initialize values here */
return module_mem;


in main.c



(#includes)
Module *module;

void main()
module = module_init();



declaration in module



opaque struct, allocated by linker, only a predefined number of instances



keep the whole struct and memory internal to the module and never expose a handler or struct.



(#includes)

void main()
module_init(_no_param_or_index_if_multiple_instances_possible_);




Is there an option to combine these somehow for opaque struct, linker instead of heap allocation and multiple/any number of instances?



solution



as proposed in some answers below, I think the best way is to:



  1. reserve space for MODULE_MAX_INSTANCE_COUNT modules in the modules source file

  2. do not define MODULE_MAX_INSTANCE_COUNT in the module itsself

  3. add an #ifndef MODULE_MAX_INSTANCE_COUNT #error to the modules header file to make sure the modules user is aware of this limitation and defines the maximum number of instances wanted for the application

  4. on initialization of an instance return either the memory address (*void) of the desricptive struct or the modules index (whatever you like more)






share|improve this question














modular approaches are pretty handy in general (portable and clean), so I try to program modules as independent of any other modules as possible. Most of my approaches are based on a struct that describes the module itself. An initialization function sets the primary parameters, afterwards a handler (pointer to desriptive struct) is passed to whatever function inside the module is called.



Right now, I am wondering what the best approach of allocation memory for the struct describing a module may be. If possible, I'd like the following:



  • Opaque struct, so the struct may only be altered by the use of provided interface functions

  • Multiple instances

  • memory allocated by linker

I see the following possibilities, that all conflict with one of my goals:



global declaration



multiple instances, allcoted by linker, but struct is not opaque



(#includes)
module_struct module;

void main()
module_init(&module);



malloc



opaque struct, multiple instances, but allcotion on heap



in module.h:



typedef module_struct Module;


in module.c init function, malloc and return pointer to allocated memory



module_mem = malloc(sizeof(module_struct ));
/* initialize values here */
return module_mem;


in main.c



(#includes)
Module *module;

void main()
module = module_init();



declaration in module



opaque struct, allocated by linker, only a predefined number of instances



keep the whole struct and memory internal to the module and never expose a handler or struct.



(#includes)

void main()
module_init(_no_param_or_index_if_multiple_instances_possible_);




Is there an option to combine these somehow for opaque struct, linker instead of heap allocation and multiple/any number of instances?



solution



as proposed in some answers below, I think the best way is to:



  1. reserve space for MODULE_MAX_INSTANCE_COUNT modules in the modules source file

  2. do not define MODULE_MAX_INSTANCE_COUNT in the module itsself

  3. add an #ifndef MODULE_MAX_INSTANCE_COUNT #error to the modules header file to make sure the modules user is aware of this limitation and defines the maximum number of instances wanted for the application

  4. on initialization of an instance return either the memory address (*void) of the desricptive struct or the modules index (whatever you like more)








share|improve this question













share|improve this question




share|improve this question








edited Sep 1 at 10:15

























asked Aug 15 at 14:22









L. Heinrichs

343111




343111







  • 12




    Most embedded FW designers are avoiding the dynamic allocation to keep the memory usage deterministic and simple. Especially if it is bare-metal and does not have underlying OS to manage the memory.
    – Eugene Sh.
    Aug 15 at 14:25











  • Exactly, thats why I want the linker to do the allocations.
    – L. Heinrichs
    Aug 15 at 14:28






  • 4




    I'm not quite sure I understand... How could you have the memory allocated by the linker if you have a dynamic number of instances? That seems quite orthogonal to me.
    – jcaron
    Aug 15 at 17:13










  • Why not let the linker allocate one big memory pool, and do your own allocations from that, which also gives you the benefit of a zero-overhead allocator. You can make the pool object static to the file with the allocation function so it's private. In some of my code, I do all the allocations in the various init routines, then I print out afterward how much was allocated, so in the final production compile I set the pool to that exact size.
    – Lee Daniel Crocker
    Aug 15 at 23:58







  • 2




    If it's a compile-time decision, you could simply define the number in your Makefile or equivalent, and you're all set. The number would not be in the source of the module, but would be application specific, and you just use an instance number as a parameter.
    – jcaron
    Aug 16 at 14:49












  • 12




    Most embedded FW designers are avoiding the dynamic allocation to keep the memory usage deterministic and simple. Especially if it is bare-metal and does not have underlying OS to manage the memory.
    – Eugene Sh.
    Aug 15 at 14:25











  • Exactly, thats why I want the linker to do the allocations.
    – L. Heinrichs
    Aug 15 at 14:28






  • 4




    I'm not quite sure I understand... How could you have the memory allocated by the linker if you have a dynamic number of instances? That seems quite orthogonal to me.
    – jcaron
    Aug 15 at 17:13










  • Why not let the linker allocate one big memory pool, and do your own allocations from that, which also gives you the benefit of a zero-overhead allocator. You can make the pool object static to the file with the allocation function so it's private. In some of my code, I do all the allocations in the various init routines, then I print out afterward how much was allocated, so in the final production compile I set the pool to that exact size.
    – Lee Daniel Crocker
    Aug 15 at 23:58







  • 2




    If it's a compile-time decision, you could simply define the number in your Makefile or equivalent, and you're all set. The number would not be in the source of the module, but would be application specific, and you just use an instance number as a parameter.
    – jcaron
    Aug 16 at 14:49







12




12




Most embedded FW designers are avoiding the dynamic allocation to keep the memory usage deterministic and simple. Especially if it is bare-metal and does not have underlying OS to manage the memory.
– Eugene Sh.
Aug 15 at 14:25





Most embedded FW designers are avoiding the dynamic allocation to keep the memory usage deterministic and simple. Especially if it is bare-metal and does not have underlying OS to manage the memory.
– Eugene Sh.
Aug 15 at 14:25













Exactly, thats why I want the linker to do the allocations.
– L. Heinrichs
Aug 15 at 14:28




Exactly, thats why I want the linker to do the allocations.
– L. Heinrichs
Aug 15 at 14:28




4




4




I'm not quite sure I understand... How could you have the memory allocated by the linker if you have a dynamic number of instances? That seems quite orthogonal to me.
– jcaron
Aug 15 at 17:13




I'm not quite sure I understand... How could you have the memory allocated by the linker if you have a dynamic number of instances? That seems quite orthogonal to me.
– jcaron
Aug 15 at 17:13












Why not let the linker allocate one big memory pool, and do your own allocations from that, which also gives you the benefit of a zero-overhead allocator. You can make the pool object static to the file with the allocation function so it's private. In some of my code, I do all the allocations in the various init routines, then I print out afterward how much was allocated, so in the final production compile I set the pool to that exact size.
– Lee Daniel Crocker
Aug 15 at 23:58





Why not let the linker allocate one big memory pool, and do your own allocations from that, which also gives you the benefit of a zero-overhead allocator. You can make the pool object static to the file with the allocation function so it's private. In some of my code, I do all the allocations in the various init routines, then I print out afterward how much was allocated, so in the final production compile I set the pool to that exact size.
– Lee Daniel Crocker
Aug 15 at 23:58





2




2




If it's a compile-time decision, you could simply define the number in your Makefile or equivalent, and you're all set. The number would not be in the source of the module, but would be application specific, and you just use an instance number as a parameter.
– jcaron
Aug 16 at 14:49




If it's a compile-time decision, you could simply define the number in your Makefile or equivalent, and you're all set. The number would not be in the source of the module, but would be application specific, and you just use an instance number as a parameter.
– jcaron
Aug 16 at 14:49










5 Answers
5






active

oldest

votes

















up vote
3
down vote



accepted











Is there an option to combine these somehow for anonymous struct,
linker instead of heap allocation and multiple/any number of
instances?




Sure there is. First, however, recognize that the "any number" of instances must be fixed, or at least an upper bound established, at compile time. This is a prerequisite for the instances to be statically allocated (what you're calling "linker allocation"). You might make the number adjustable without source modification by declaring a macro that specifies it.



Then the source file containing the actual struct declaration and all its associated functions also declares an array of instances with internal linkage. It provides either an array, with external linkage, of pointers to the instances or else a function for accessing the various pointers by index. The function variation is a bit more modular:



module.c



#include <module.h>

// 4 instances by default; can be overridden at compile time
#ifndef NUM_MODULE_INSTANCES
#define NUM_MODULE_INSTANCES 4
#endif

struct module
int demo;
;

// has internal linkage, so is not directly visible from other files:
static struct module instances[NUM_MODULE_INSTANCES];

// module functions

struct module *module_init(unsigned index)
instances[index].demo = 42;
return &instances[index];



I guess you're already familiar with how the header would then declare the struct as an incomplete type and declare all the functions (written in terms of pointers to that type). For example:



module.h



#ifndef MODULE_H
#define MODULE_H

struct module;

struct module *module_init(unsigned index);

// other functions ...

#endif


Now struct module is opaque in translation units other than module.c,* and you can access and use up to the number of instances defined at compile time without any dynamic allocation.




*Unless you copy its definition, of course. The point is that module.h does not do that.






share|improve this answer




















  • I think it's an odd design to pass the index from outside the class. When I implement memory pools like this, I let the index be a private counter, increasing by 1 for every allocated instance. Until you reach "NUM_MODULE_INSTANCES", where the constructor will return an out of memory error.
    – Lundin
    Aug 17 at 6:16











  • That's a fair point, @Lundin. That aspect of the design supposes that the indexes have inherent significance, which may or may not in fact be the case. It is the case, albeit trivially so, for the OP's starting case. Such significance, if it exists, could be further supported by providing a means to obtain an instance pointer without initializing.
    – John Bollinger
    Aug 17 at 11:51










  • So basically you reserve memory for n modules, no matter how many will be used, then return a pointer to the next unused element if the application initializes it. I guess that could work.
    – L. Heinrichs
    Aug 17 at 17:05










  • @L.Heinrichs Yes, since embedded systems are of a deterministic nature. There's no such thing as "endless amount of objects" nor "unknown amount of objects". Objects are often singletons too (hardware drivers), so there's often no need for the memory pool, as just one single instance of the object will exist.
    – Lundin
    Aug 20 at 14:06










  • I agree for most cases. The question had some theoretical interest as well. But I could use hundreds of 1-wire temperature sensors if there are enough IOs available (as the one example I can come up with now).
    – L. Heinrichs
    Aug 20 at 15:08

















up vote
22
down vote













I program small micro-controllers in C++, which achieves exactly what you want.



What you call a module is a C++ class, it can contain data (either externally accessible or not) and functions (likewise). The constructor (a dedicated function) initilializes it. The constructor can take run-time parameters or (my favourite) compile-time (template) parameters. The functions within the class implicitly get the class variable as first parameter. (Or, often my preference, the class can act as a hidden singleton, so all data is accessed without this overhead).



The class object can be global (so you know at link-time that everything will fit), or stack-local, presumably in the main. (I don't like C++ globals because of the undefined global initialization order, so I prefer stack-local).



My preferred programming style is that modules are static classes, and their (static) configuration is by template parameters. This avoids nearly all overhad and enables optimization. Combine this with a tool that calculates the stack size and you can sleep without worries :)



My talk on this way of coding in C++: Objects? No Thanks!



A lot of embedded / microcontroller programmers seem to dislike C++ because they think it would force them to use all of C++. That is absolutely not neccessary, and would be a very bad idea. (You probably don't use all of C either! Think heap, floating point, setjmp/longjmp, printf, ...)




In a comment Adam Haun mentions RAII and initialization. IMO RAII has more to do with deconstruction, but his point is valid: global objects will be constructed before your main starts, so they might work on invalid assumptions (like a main clock speed that will be changed lateron). That is one more reason NOT to use global code-initialized objects. (I use a linker script that will fail when I have global code-initialized objects.) IMO such 'objects' should be explicitly created and passed around. This includes a 'waiting' facility 'object' that provides a wait() function. In my setup this is 'object' that sets the clock speed of the chip.



Talking about RAII: that is one more C++ feature that is very usefull in small embedded systems, although not for the reason (memory deallocation) it most used for in largere systems (small embedded systems mostly don't use dynamic memory deallocation). Think of locking a resource: you can make the locked resource a wrapper object, and restrict access to the resource to be only possible via the locking wrapper. When the wrapper goes out of scope, the resource is unlocked. This prevents access without locking, and makes it much more unlikely to forget the unlocking. with some (template) magic it can be zero-overhead.




The original question didn't mention C, hence my C++-centric answer. If it realy must be C....



You could use macro trickery: declare your stucts publicly, so they have a type and can be allocated globally, but mangle the names of their components beyond usability, unless some macro is defined differently, which is the case in your module's .c file. For extra security you could use the compile time in the mangling.



Or have a public version of your struct that has nothing usefull in it, and have the private version (with usefull data) only in your .c file, and assert that they are the same size. A bit of make-file trickery could automate this.




@Lundins comment about bad (embedded) programmers:



  • The type of programmer you describe would probably make a mess in any language. Macro's (present in C and C++) are one obvious way.


  • Tooling can help to some extent. For my students I mandate a built script that specifies no-exceptions, no-rtti, and gives a linker error when either the heap is used or code-initialized globals are present. And it specifies warning=error and enables nearly all warnings.


  • I encourage using templates, but with constexpr and concepts metaprogramming is less and less required.


  • "confused Arduino programmers" I would very much like to replace the Arduino (wiring, replication of code in libraries) programming style with a modern C++ approach, which can be easiere, more secure, and produce faster and smaller code. If only I had the time and power....






share|improve this answer






















  • Thanks for this answer! Using C++ is an option, but we are using C in my company (what I havent mentioned explicitly). I have updated the question to let people know Im talking about C.
    – L. Heinrichs
    Aug 15 at 14:51










  • Why are you using (only) C? Maybe this gives you chance to convince them to at least consider C++... What you want is essentialy (a small part of) C++ realised in C.
    – Wouter van Ooijen
    Aug 15 at 14:55











  • What I do in my (first 'real' embedded hobby project) is initializing simple default in the constructor, and use a separate Init method for relevant classes. Another benefit is that I can pass stub pointers for unit testing.
    – Michel Keijzers
    Aug 15 at 15:00






  • 2




    @Michel for a hobby project you are free to choose the language? Take C++!
    – Wouter van Ooijen
    Aug 15 at 15:02






  • 4




    And while it is indeed perfectly possible to write good C++ programs for embedded, the problem is that some >50% of all embedded systems programmers out there are quacks, confused PC programmers, Arduino hobbyists etc etc. These kind of people simply cannot use a clean subset of C++, even if you explain it to their face. Give them C++ and before you know it, they will use the whole of STL, template metaprogramming, exception handling, multiple inheritance and so on. And the result is of course complete trash. This is, sadly, how some 8 out of 10 embedded C++ projects end up.
    – Lundin
    Aug 16 at 15:09


















up vote
7
down vote













I believe FreeRTOS (maybe another OS?) does something like what you're looking for by defining 2 different versions of the struct.

The 'real' one, used internally by the OS functions, and a 'fake' one which is the same size as the 'real' one, but doesn't have any useful members inside (just a bunch of int dummy1 and similar).

Only the 'fake' struct is exposed outside of the OS code, and this is used to allocate memory to static instances of the struct.

Internally, when functions in the OS are called, they are passed the address of the external 'fake' struct as a handle, and this is then typecast as a pointer to a 'real' struct so the OS functions can do what they need to do.






share|improve this answer






















  • Good idea, I guess i could use --- #define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)])) --- BUILD_BUG_ON(sizeof(real_struct) != sizeof(fake_struct)) ----
    – L. Heinrichs
    Aug 15 at 14:47

















up vote
2
down vote














Anonymous struct, so the struct may only be altered by the use of provided interface functions




In my opinion, this is pointless. You can put a comment there, but no point trying to hide it further.



C will never provide such high isolation, even if there is no declaration for the struct, it will be easy to accidentally overwrite it with e.g. mistaken memcpy() or buffer overflow.



Instead, just give the struct a name and trust other people to write good code also. It will also make debugging easier when the struct has a name you can use to refer to it.






share|improve this answer



























    up vote
    2
    down vote













    Pure SW questions are better asked at https://stackoverflow.com.



    The concept with exposing a struct of incomplete type to caller, like you describe, is often called "opaque type" or "opaque pointers" - anonymous struct means something else entirely.



    The problem with this is that the caller won't be able to allocate instances of the object, only pointers to it. On a PC you would use malloc inside the objects "constructor", but malloc is a no-go in embedded systems.



    So what you do in embedded is to provide a memory pool. You have a limited amount of RAM, so restricting the number of objects that can be created is usually not a problem.



    See Static allocation of opaque data types over at SO.






    share|improve this answer




















    • Ou thank you for clarifying the naming confusion on my end, ill adjust the OP. I was thinking of going to stack overflow, but decided i would like to target embedded programmers specifically.
      – L. Heinrichs
      Aug 17 at 17:04










    Your Answer




    StackExchange.ifUsing("editor", function ()
    return StackExchange.using("mathjaxEditing", function ()
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["\$", "\$"]]);
    );
    );
    , "mathjax-editing");

    StackExchange.ifUsing("editor", function ()
    return StackExchange.using("schematics", function ()
    StackExchange.schematics.init();
    );
    , "cicuitlab");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "135"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    convertImagesToLinks: false,
    noModals: false,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













     

    draft saved


    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2felectronics.stackexchange.com%2fquestions%2f391178%2fpossibilities-for-allocating-memory-for-modular-firmware-design-in-c%23new-answer', 'question_page');

    );

    Post as a guest






























    5 Answers
    5






    active

    oldest

    votes








    5 Answers
    5






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    3
    down vote



    accepted











    Is there an option to combine these somehow for anonymous struct,
    linker instead of heap allocation and multiple/any number of
    instances?




    Sure there is. First, however, recognize that the "any number" of instances must be fixed, or at least an upper bound established, at compile time. This is a prerequisite for the instances to be statically allocated (what you're calling "linker allocation"). You might make the number adjustable without source modification by declaring a macro that specifies it.



    Then the source file containing the actual struct declaration and all its associated functions also declares an array of instances with internal linkage. It provides either an array, with external linkage, of pointers to the instances or else a function for accessing the various pointers by index. The function variation is a bit more modular:



    module.c



    #include <module.h>

    // 4 instances by default; can be overridden at compile time
    #ifndef NUM_MODULE_INSTANCES
    #define NUM_MODULE_INSTANCES 4
    #endif

    struct module
    int demo;
    ;

    // has internal linkage, so is not directly visible from other files:
    static struct module instances[NUM_MODULE_INSTANCES];

    // module functions

    struct module *module_init(unsigned index)
    instances[index].demo = 42;
    return &instances[index];



    I guess you're already familiar with how the header would then declare the struct as an incomplete type and declare all the functions (written in terms of pointers to that type). For example:



    module.h



    #ifndef MODULE_H
    #define MODULE_H

    struct module;

    struct module *module_init(unsigned index);

    // other functions ...

    #endif


    Now struct module is opaque in translation units other than module.c,* and you can access and use up to the number of instances defined at compile time without any dynamic allocation.




    *Unless you copy its definition, of course. The point is that module.h does not do that.






    share|improve this answer




















    • I think it's an odd design to pass the index from outside the class. When I implement memory pools like this, I let the index be a private counter, increasing by 1 for every allocated instance. Until you reach "NUM_MODULE_INSTANCES", where the constructor will return an out of memory error.
      – Lundin
      Aug 17 at 6:16











    • That's a fair point, @Lundin. That aspect of the design supposes that the indexes have inherent significance, which may or may not in fact be the case. It is the case, albeit trivially so, for the OP's starting case. Such significance, if it exists, could be further supported by providing a means to obtain an instance pointer without initializing.
      – John Bollinger
      Aug 17 at 11:51










    • So basically you reserve memory for n modules, no matter how many will be used, then return a pointer to the next unused element if the application initializes it. I guess that could work.
      – L. Heinrichs
      Aug 17 at 17:05










    • @L.Heinrichs Yes, since embedded systems are of a deterministic nature. There's no such thing as "endless amount of objects" nor "unknown amount of objects". Objects are often singletons too (hardware drivers), so there's often no need for the memory pool, as just one single instance of the object will exist.
      – Lundin
      Aug 20 at 14:06










    • I agree for most cases. The question had some theoretical interest as well. But I could use hundreds of 1-wire temperature sensors if there are enough IOs available (as the one example I can come up with now).
      – L. Heinrichs
      Aug 20 at 15:08














    up vote
    3
    down vote



    accepted











    Is there an option to combine these somehow for anonymous struct,
    linker instead of heap allocation and multiple/any number of
    instances?




    Sure there is. First, however, recognize that the "any number" of instances must be fixed, or at least an upper bound established, at compile time. This is a prerequisite for the instances to be statically allocated (what you're calling "linker allocation"). You might make the number adjustable without source modification by declaring a macro that specifies it.



    Then the source file containing the actual struct declaration and all its associated functions also declares an array of instances with internal linkage. It provides either an array, with external linkage, of pointers to the instances or else a function for accessing the various pointers by index. The function variation is a bit more modular:



    module.c



    #include <module.h>

    // 4 instances by default; can be overridden at compile time
    #ifndef NUM_MODULE_INSTANCES
    #define NUM_MODULE_INSTANCES 4
    #endif

    struct module
    int demo;
    ;

    // has internal linkage, so is not directly visible from other files:
    static struct module instances[NUM_MODULE_INSTANCES];

    // module functions

    struct module *module_init(unsigned index)
    instances[index].demo = 42;
    return &instances[index];



    I guess you're already familiar with how the header would then declare the struct as an incomplete type and declare all the functions (written in terms of pointers to that type). For example:



    module.h



    #ifndef MODULE_H
    #define MODULE_H

    struct module;

    struct module *module_init(unsigned index);

    // other functions ...

    #endif


    Now struct module is opaque in translation units other than module.c,* and you can access and use up to the number of instances defined at compile time without any dynamic allocation.




    *Unless you copy its definition, of course. The point is that module.h does not do that.






    share|improve this answer




















    • I think it's an odd design to pass the index from outside the class. When I implement memory pools like this, I let the index be a private counter, increasing by 1 for every allocated instance. Until you reach "NUM_MODULE_INSTANCES", where the constructor will return an out of memory error.
      – Lundin
      Aug 17 at 6:16











    • That's a fair point, @Lundin. That aspect of the design supposes that the indexes have inherent significance, which may or may not in fact be the case. It is the case, albeit trivially so, for the OP's starting case. Such significance, if it exists, could be further supported by providing a means to obtain an instance pointer without initializing.
      – John Bollinger
      Aug 17 at 11:51










    • So basically you reserve memory for n modules, no matter how many will be used, then return a pointer to the next unused element if the application initializes it. I guess that could work.
      – L. Heinrichs
      Aug 17 at 17:05










    • @L.Heinrichs Yes, since embedded systems are of a deterministic nature. There's no such thing as "endless amount of objects" nor "unknown amount of objects". Objects are often singletons too (hardware drivers), so there's often no need for the memory pool, as just one single instance of the object will exist.
      – Lundin
      Aug 20 at 14:06










    • I agree for most cases. The question had some theoretical interest as well. But I could use hundreds of 1-wire temperature sensors if there are enough IOs available (as the one example I can come up with now).
      – L. Heinrichs
      Aug 20 at 15:08












    up vote
    3
    down vote



    accepted







    up vote
    3
    down vote



    accepted







    Is there an option to combine these somehow for anonymous struct,
    linker instead of heap allocation and multiple/any number of
    instances?




    Sure there is. First, however, recognize that the "any number" of instances must be fixed, or at least an upper bound established, at compile time. This is a prerequisite for the instances to be statically allocated (what you're calling "linker allocation"). You might make the number adjustable without source modification by declaring a macro that specifies it.



    Then the source file containing the actual struct declaration and all its associated functions also declares an array of instances with internal linkage. It provides either an array, with external linkage, of pointers to the instances or else a function for accessing the various pointers by index. The function variation is a bit more modular:



    module.c



    #include <module.h>

    // 4 instances by default; can be overridden at compile time
    #ifndef NUM_MODULE_INSTANCES
    #define NUM_MODULE_INSTANCES 4
    #endif

    struct module
    int demo;
    ;

    // has internal linkage, so is not directly visible from other files:
    static struct module instances[NUM_MODULE_INSTANCES];

    // module functions

    struct module *module_init(unsigned index)
    instances[index].demo = 42;
    return &instances[index];



    I guess you're already familiar with how the header would then declare the struct as an incomplete type and declare all the functions (written in terms of pointers to that type). For example:



    module.h



    #ifndef MODULE_H
    #define MODULE_H

    struct module;

    struct module *module_init(unsigned index);

    // other functions ...

    #endif


    Now struct module is opaque in translation units other than module.c,* and you can access and use up to the number of instances defined at compile time without any dynamic allocation.




    *Unless you copy its definition, of course. The point is that module.h does not do that.






    share|improve this answer













    Is there an option to combine these somehow for anonymous struct,
    linker instead of heap allocation and multiple/any number of
    instances?




    Sure there is. First, however, recognize that the "any number" of instances must be fixed, or at least an upper bound established, at compile time. This is a prerequisite for the instances to be statically allocated (what you're calling "linker allocation"). You might make the number adjustable without source modification by declaring a macro that specifies it.



    Then the source file containing the actual struct declaration and all its associated functions also declares an array of instances with internal linkage. It provides either an array, with external linkage, of pointers to the instances or else a function for accessing the various pointers by index. The function variation is a bit more modular:



    module.c



    #include <module.h>

    // 4 instances by default; can be overridden at compile time
    #ifndef NUM_MODULE_INSTANCES
    #define NUM_MODULE_INSTANCES 4
    #endif

    struct module
    int demo;
    ;

    // has internal linkage, so is not directly visible from other files:
    static struct module instances[NUM_MODULE_INSTANCES];

    // module functions

    struct module *module_init(unsigned index)
    instances[index].demo = 42;
    return &instances[index];



    I guess you're already familiar with how the header would then declare the struct as an incomplete type and declare all the functions (written in terms of pointers to that type). For example:



    module.h



    #ifndef MODULE_H
    #define MODULE_H

    struct module;

    struct module *module_init(unsigned index);

    // other functions ...

    #endif


    Now struct module is opaque in translation units other than module.c,* and you can access and use up to the number of instances defined at compile time without any dynamic allocation.




    *Unless you copy its definition, of course. The point is that module.h does not do that.







    share|improve this answer












    share|improve this answer



    share|improve this answer










    answered Aug 16 at 15:25









    John Bollinger

    1461




    1461











    • I think it's an odd design to pass the index from outside the class. When I implement memory pools like this, I let the index be a private counter, increasing by 1 for every allocated instance. Until you reach "NUM_MODULE_INSTANCES", where the constructor will return an out of memory error.
      – Lundin
      Aug 17 at 6:16











    • That's a fair point, @Lundin. That aspect of the design supposes that the indexes have inherent significance, which may or may not in fact be the case. It is the case, albeit trivially so, for the OP's starting case. Such significance, if it exists, could be further supported by providing a means to obtain an instance pointer without initializing.
      – John Bollinger
      Aug 17 at 11:51










    • So basically you reserve memory for n modules, no matter how many will be used, then return a pointer to the next unused element if the application initializes it. I guess that could work.
      – L. Heinrichs
      Aug 17 at 17:05










    • @L.Heinrichs Yes, since embedded systems are of a deterministic nature. There's no such thing as "endless amount of objects" nor "unknown amount of objects". Objects are often singletons too (hardware drivers), so there's often no need for the memory pool, as just one single instance of the object will exist.
      – Lundin
      Aug 20 at 14:06










    • I agree for most cases. The question had some theoretical interest as well. But I could use hundreds of 1-wire temperature sensors if there are enough IOs available (as the one example I can come up with now).
      – L. Heinrichs
      Aug 20 at 15:08
















    • I think it's an odd design to pass the index from outside the class. When I implement memory pools like this, I let the index be a private counter, increasing by 1 for every allocated instance. Until you reach "NUM_MODULE_INSTANCES", where the constructor will return an out of memory error.
      – Lundin
      Aug 17 at 6:16











    • That's a fair point, @Lundin. That aspect of the design supposes that the indexes have inherent significance, which may or may not in fact be the case. It is the case, albeit trivially so, for the OP's starting case. Such significance, if it exists, could be further supported by providing a means to obtain an instance pointer without initializing.
      – John Bollinger
      Aug 17 at 11:51










    • So basically you reserve memory for n modules, no matter how many will be used, then return a pointer to the next unused element if the application initializes it. I guess that could work.
      – L. Heinrichs
      Aug 17 at 17:05










    • @L.Heinrichs Yes, since embedded systems are of a deterministic nature. There's no such thing as "endless amount of objects" nor "unknown amount of objects". Objects are often singletons too (hardware drivers), so there's often no need for the memory pool, as just one single instance of the object will exist.
      – Lundin
      Aug 20 at 14:06










    • I agree for most cases. The question had some theoretical interest as well. But I could use hundreds of 1-wire temperature sensors if there are enough IOs available (as the one example I can come up with now).
      – L. Heinrichs
      Aug 20 at 15:08















    I think it's an odd design to pass the index from outside the class. When I implement memory pools like this, I let the index be a private counter, increasing by 1 for every allocated instance. Until you reach "NUM_MODULE_INSTANCES", where the constructor will return an out of memory error.
    – Lundin
    Aug 17 at 6:16





    I think it's an odd design to pass the index from outside the class. When I implement memory pools like this, I let the index be a private counter, increasing by 1 for every allocated instance. Until you reach "NUM_MODULE_INSTANCES", where the constructor will return an out of memory error.
    – Lundin
    Aug 17 at 6:16













    That's a fair point, @Lundin. That aspect of the design supposes that the indexes have inherent significance, which may or may not in fact be the case. It is the case, albeit trivially so, for the OP's starting case. Such significance, if it exists, could be further supported by providing a means to obtain an instance pointer without initializing.
    – John Bollinger
    Aug 17 at 11:51




    That's a fair point, @Lundin. That aspect of the design supposes that the indexes have inherent significance, which may or may not in fact be the case. It is the case, albeit trivially so, for the OP's starting case. Such significance, if it exists, could be further supported by providing a means to obtain an instance pointer without initializing.
    – John Bollinger
    Aug 17 at 11:51












    So basically you reserve memory for n modules, no matter how many will be used, then return a pointer to the next unused element if the application initializes it. I guess that could work.
    – L. Heinrichs
    Aug 17 at 17:05




    So basically you reserve memory for n modules, no matter how many will be used, then return a pointer to the next unused element if the application initializes it. I guess that could work.
    – L. Heinrichs
    Aug 17 at 17:05












    @L.Heinrichs Yes, since embedded systems are of a deterministic nature. There's no such thing as "endless amount of objects" nor "unknown amount of objects". Objects are often singletons too (hardware drivers), so there's often no need for the memory pool, as just one single instance of the object will exist.
    – Lundin
    Aug 20 at 14:06




    @L.Heinrichs Yes, since embedded systems are of a deterministic nature. There's no such thing as "endless amount of objects" nor "unknown amount of objects". Objects are often singletons too (hardware drivers), so there's often no need for the memory pool, as just one single instance of the object will exist.
    – Lundin
    Aug 20 at 14:06












    I agree for most cases. The question had some theoretical interest as well. But I could use hundreds of 1-wire temperature sensors if there are enough IOs available (as the one example I can come up with now).
    – L. Heinrichs
    Aug 20 at 15:08




    I agree for most cases. The question had some theoretical interest as well. But I could use hundreds of 1-wire temperature sensors if there are enough IOs available (as the one example I can come up with now).
    – L. Heinrichs
    Aug 20 at 15:08












    up vote
    22
    down vote













    I program small micro-controllers in C++, which achieves exactly what you want.



    What you call a module is a C++ class, it can contain data (either externally accessible or not) and functions (likewise). The constructor (a dedicated function) initilializes it. The constructor can take run-time parameters or (my favourite) compile-time (template) parameters. The functions within the class implicitly get the class variable as first parameter. (Or, often my preference, the class can act as a hidden singleton, so all data is accessed without this overhead).



    The class object can be global (so you know at link-time that everything will fit), or stack-local, presumably in the main. (I don't like C++ globals because of the undefined global initialization order, so I prefer stack-local).



    My preferred programming style is that modules are static classes, and their (static) configuration is by template parameters. This avoids nearly all overhad and enables optimization. Combine this with a tool that calculates the stack size and you can sleep without worries :)



    My talk on this way of coding in C++: Objects? No Thanks!



    A lot of embedded / microcontroller programmers seem to dislike C++ because they think it would force them to use all of C++. That is absolutely not neccessary, and would be a very bad idea. (You probably don't use all of C either! Think heap, floating point, setjmp/longjmp, printf, ...)




    In a comment Adam Haun mentions RAII and initialization. IMO RAII has more to do with deconstruction, but his point is valid: global objects will be constructed before your main starts, so they might work on invalid assumptions (like a main clock speed that will be changed lateron). That is one more reason NOT to use global code-initialized objects. (I use a linker script that will fail when I have global code-initialized objects.) IMO such 'objects' should be explicitly created and passed around. This includes a 'waiting' facility 'object' that provides a wait() function. In my setup this is 'object' that sets the clock speed of the chip.



    Talking about RAII: that is one more C++ feature that is very usefull in small embedded systems, although not for the reason (memory deallocation) it most used for in largere systems (small embedded systems mostly don't use dynamic memory deallocation). Think of locking a resource: you can make the locked resource a wrapper object, and restrict access to the resource to be only possible via the locking wrapper. When the wrapper goes out of scope, the resource is unlocked. This prevents access without locking, and makes it much more unlikely to forget the unlocking. with some (template) magic it can be zero-overhead.




    The original question didn't mention C, hence my C++-centric answer. If it realy must be C....



    You could use macro trickery: declare your stucts publicly, so they have a type and can be allocated globally, but mangle the names of their components beyond usability, unless some macro is defined differently, which is the case in your module's .c file. For extra security you could use the compile time in the mangling.



    Or have a public version of your struct that has nothing usefull in it, and have the private version (with usefull data) only in your .c file, and assert that they are the same size. A bit of make-file trickery could automate this.




    @Lundins comment about bad (embedded) programmers:



    • The type of programmer you describe would probably make a mess in any language. Macro's (present in C and C++) are one obvious way.


    • Tooling can help to some extent. For my students I mandate a built script that specifies no-exceptions, no-rtti, and gives a linker error when either the heap is used or code-initialized globals are present. And it specifies warning=error and enables nearly all warnings.


    • I encourage using templates, but with constexpr and concepts metaprogramming is less and less required.


    • "confused Arduino programmers" I would very much like to replace the Arduino (wiring, replication of code in libraries) programming style with a modern C++ approach, which can be easiere, more secure, and produce faster and smaller code. If only I had the time and power....






    share|improve this answer






















    • Thanks for this answer! Using C++ is an option, but we are using C in my company (what I havent mentioned explicitly). I have updated the question to let people know Im talking about C.
      – L. Heinrichs
      Aug 15 at 14:51










    • Why are you using (only) C? Maybe this gives you chance to convince them to at least consider C++... What you want is essentialy (a small part of) C++ realised in C.
      – Wouter van Ooijen
      Aug 15 at 14:55











    • What I do in my (first 'real' embedded hobby project) is initializing simple default in the constructor, and use a separate Init method for relevant classes. Another benefit is that I can pass stub pointers for unit testing.
      – Michel Keijzers
      Aug 15 at 15:00






    • 2




      @Michel for a hobby project you are free to choose the language? Take C++!
      – Wouter van Ooijen
      Aug 15 at 15:02






    • 4




      And while it is indeed perfectly possible to write good C++ programs for embedded, the problem is that some >50% of all embedded systems programmers out there are quacks, confused PC programmers, Arduino hobbyists etc etc. These kind of people simply cannot use a clean subset of C++, even if you explain it to their face. Give them C++ and before you know it, they will use the whole of STL, template metaprogramming, exception handling, multiple inheritance and so on. And the result is of course complete trash. This is, sadly, how some 8 out of 10 embedded C++ projects end up.
      – Lundin
      Aug 16 at 15:09















    up vote
    22
    down vote













    I program small micro-controllers in C++, which achieves exactly what you want.



    What you call a module is a C++ class, it can contain data (either externally accessible or not) and functions (likewise). The constructor (a dedicated function) initilializes it. The constructor can take run-time parameters or (my favourite) compile-time (template) parameters. The functions within the class implicitly get the class variable as first parameter. (Or, often my preference, the class can act as a hidden singleton, so all data is accessed without this overhead).



    The class object can be global (so you know at link-time that everything will fit), or stack-local, presumably in the main. (I don't like C++ globals because of the undefined global initialization order, so I prefer stack-local).



    My preferred programming style is that modules are static classes, and their (static) configuration is by template parameters. This avoids nearly all overhad and enables optimization. Combine this with a tool that calculates the stack size and you can sleep without worries :)



    My talk on this way of coding in C++: Objects? No Thanks!



    A lot of embedded / microcontroller programmers seem to dislike C++ because they think it would force them to use all of C++. That is absolutely not neccessary, and would be a very bad idea. (You probably don't use all of C either! Think heap, floating point, setjmp/longjmp, printf, ...)




    In a comment Adam Haun mentions RAII and initialization. IMO RAII has more to do with deconstruction, but his point is valid: global objects will be constructed before your main starts, so they might work on invalid assumptions (like a main clock speed that will be changed lateron). That is one more reason NOT to use global code-initialized objects. (I use a linker script that will fail when I have global code-initialized objects.) IMO such 'objects' should be explicitly created and passed around. This includes a 'waiting' facility 'object' that provides a wait() function. In my setup this is 'object' that sets the clock speed of the chip.



    Talking about RAII: that is one more C++ feature that is very usefull in small embedded systems, although not for the reason (memory deallocation) it most used for in largere systems (small embedded systems mostly don't use dynamic memory deallocation). Think of locking a resource: you can make the locked resource a wrapper object, and restrict access to the resource to be only possible via the locking wrapper. When the wrapper goes out of scope, the resource is unlocked. This prevents access without locking, and makes it much more unlikely to forget the unlocking. with some (template) magic it can be zero-overhead.




    The original question didn't mention C, hence my C++-centric answer. If it realy must be C....



    You could use macro trickery: declare your stucts publicly, so they have a type and can be allocated globally, but mangle the names of their components beyond usability, unless some macro is defined differently, which is the case in your module's .c file. For extra security you could use the compile time in the mangling.



    Or have a public version of your struct that has nothing usefull in it, and have the private version (with usefull data) only in your .c file, and assert that they are the same size. A bit of make-file trickery could automate this.




    @Lundins comment about bad (embedded) programmers:



    • The type of programmer you describe would probably make a mess in any language. Macro's (present in C and C++) are one obvious way.


    • Tooling can help to some extent. For my students I mandate a built script that specifies no-exceptions, no-rtti, and gives a linker error when either the heap is used or code-initialized globals are present. And it specifies warning=error and enables nearly all warnings.


    • I encourage using templates, but with constexpr and concepts metaprogramming is less and less required.


    • "confused Arduino programmers" I would very much like to replace the Arduino (wiring, replication of code in libraries) programming style with a modern C++ approach, which can be easiere, more secure, and produce faster and smaller code. If only I had the time and power....






    share|improve this answer






















    • Thanks for this answer! Using C++ is an option, but we are using C in my company (what I havent mentioned explicitly). I have updated the question to let people know Im talking about C.
      – L. Heinrichs
      Aug 15 at 14:51










    • Why are you using (only) C? Maybe this gives you chance to convince them to at least consider C++... What you want is essentialy (a small part of) C++ realised in C.
      – Wouter van Ooijen
      Aug 15 at 14:55











    • What I do in my (first 'real' embedded hobby project) is initializing simple default in the constructor, and use a separate Init method for relevant classes. Another benefit is that I can pass stub pointers for unit testing.
      – Michel Keijzers
      Aug 15 at 15:00






    • 2




      @Michel for a hobby project you are free to choose the language? Take C++!
      – Wouter van Ooijen
      Aug 15 at 15:02






    • 4




      And while it is indeed perfectly possible to write good C++ programs for embedded, the problem is that some >50% of all embedded systems programmers out there are quacks, confused PC programmers, Arduino hobbyists etc etc. These kind of people simply cannot use a clean subset of C++, even if you explain it to their face. Give them C++ and before you know it, they will use the whole of STL, template metaprogramming, exception handling, multiple inheritance and so on. And the result is of course complete trash. This is, sadly, how some 8 out of 10 embedded C++ projects end up.
      – Lundin
      Aug 16 at 15:09













    up vote
    22
    down vote










    up vote
    22
    down vote









    I program small micro-controllers in C++, which achieves exactly what you want.



    What you call a module is a C++ class, it can contain data (either externally accessible or not) and functions (likewise). The constructor (a dedicated function) initilializes it. The constructor can take run-time parameters or (my favourite) compile-time (template) parameters. The functions within the class implicitly get the class variable as first parameter. (Or, often my preference, the class can act as a hidden singleton, so all data is accessed without this overhead).



    The class object can be global (so you know at link-time that everything will fit), or stack-local, presumably in the main. (I don't like C++ globals because of the undefined global initialization order, so I prefer stack-local).



    My preferred programming style is that modules are static classes, and their (static) configuration is by template parameters. This avoids nearly all overhad and enables optimization. Combine this with a tool that calculates the stack size and you can sleep without worries :)



    My talk on this way of coding in C++: Objects? No Thanks!



    A lot of embedded / microcontroller programmers seem to dislike C++ because they think it would force them to use all of C++. That is absolutely not neccessary, and would be a very bad idea. (You probably don't use all of C either! Think heap, floating point, setjmp/longjmp, printf, ...)




    In a comment Adam Haun mentions RAII and initialization. IMO RAII has more to do with deconstruction, but his point is valid: global objects will be constructed before your main starts, so they might work on invalid assumptions (like a main clock speed that will be changed lateron). That is one more reason NOT to use global code-initialized objects. (I use a linker script that will fail when I have global code-initialized objects.) IMO such 'objects' should be explicitly created and passed around. This includes a 'waiting' facility 'object' that provides a wait() function. In my setup this is 'object' that sets the clock speed of the chip.



    Talking about RAII: that is one more C++ feature that is very usefull in small embedded systems, although not for the reason (memory deallocation) it most used for in largere systems (small embedded systems mostly don't use dynamic memory deallocation). Think of locking a resource: you can make the locked resource a wrapper object, and restrict access to the resource to be only possible via the locking wrapper. When the wrapper goes out of scope, the resource is unlocked. This prevents access without locking, and makes it much more unlikely to forget the unlocking. with some (template) magic it can be zero-overhead.




    The original question didn't mention C, hence my C++-centric answer. If it realy must be C....



    You could use macro trickery: declare your stucts publicly, so they have a type and can be allocated globally, but mangle the names of their components beyond usability, unless some macro is defined differently, which is the case in your module's .c file. For extra security you could use the compile time in the mangling.



    Or have a public version of your struct that has nothing usefull in it, and have the private version (with usefull data) only in your .c file, and assert that they are the same size. A bit of make-file trickery could automate this.




    @Lundins comment about bad (embedded) programmers:



    • The type of programmer you describe would probably make a mess in any language. Macro's (present in C and C++) are one obvious way.


    • Tooling can help to some extent. For my students I mandate a built script that specifies no-exceptions, no-rtti, and gives a linker error when either the heap is used or code-initialized globals are present. And it specifies warning=error and enables nearly all warnings.


    • I encourage using templates, but with constexpr and concepts metaprogramming is less and less required.


    • "confused Arduino programmers" I would very much like to replace the Arduino (wiring, replication of code in libraries) programming style with a modern C++ approach, which can be easiere, more secure, and produce faster and smaller code. If only I had the time and power....






    share|improve this answer














    I program small micro-controllers in C++, which achieves exactly what you want.



    What you call a module is a C++ class, it can contain data (either externally accessible or not) and functions (likewise). The constructor (a dedicated function) initilializes it. The constructor can take run-time parameters or (my favourite) compile-time (template) parameters. The functions within the class implicitly get the class variable as first parameter. (Or, often my preference, the class can act as a hidden singleton, so all data is accessed without this overhead).



    The class object can be global (so you know at link-time that everything will fit), or stack-local, presumably in the main. (I don't like C++ globals because of the undefined global initialization order, so I prefer stack-local).



    My preferred programming style is that modules are static classes, and their (static) configuration is by template parameters. This avoids nearly all overhad and enables optimization. Combine this with a tool that calculates the stack size and you can sleep without worries :)



    My talk on this way of coding in C++: Objects? No Thanks!



    A lot of embedded / microcontroller programmers seem to dislike C++ because they think it would force them to use all of C++. That is absolutely not neccessary, and would be a very bad idea. (You probably don't use all of C either! Think heap, floating point, setjmp/longjmp, printf, ...)




    In a comment Adam Haun mentions RAII and initialization. IMO RAII has more to do with deconstruction, but his point is valid: global objects will be constructed before your main starts, so they might work on invalid assumptions (like a main clock speed that will be changed lateron). That is one more reason NOT to use global code-initialized objects. (I use a linker script that will fail when I have global code-initialized objects.) IMO such 'objects' should be explicitly created and passed around. This includes a 'waiting' facility 'object' that provides a wait() function. In my setup this is 'object' that sets the clock speed of the chip.



    Talking about RAII: that is one more C++ feature that is very usefull in small embedded systems, although not for the reason (memory deallocation) it most used for in largere systems (small embedded systems mostly don't use dynamic memory deallocation). Think of locking a resource: you can make the locked resource a wrapper object, and restrict access to the resource to be only possible via the locking wrapper. When the wrapper goes out of scope, the resource is unlocked. This prevents access without locking, and makes it much more unlikely to forget the unlocking. with some (template) magic it can be zero-overhead.




    The original question didn't mention C, hence my C++-centric answer. If it realy must be C....



    You could use macro trickery: declare your stucts publicly, so they have a type and can be allocated globally, but mangle the names of their components beyond usability, unless some macro is defined differently, which is the case in your module's .c file. For extra security you could use the compile time in the mangling.



    Or have a public version of your struct that has nothing usefull in it, and have the private version (with usefull data) only in your .c file, and assert that they are the same size. A bit of make-file trickery could automate this.




    @Lundins comment about bad (embedded) programmers:



    • The type of programmer you describe would probably make a mess in any language. Macro's (present in C and C++) are one obvious way.


    • Tooling can help to some extent. For my students I mandate a built script that specifies no-exceptions, no-rtti, and gives a linker error when either the heap is used or code-initialized globals are present. And it specifies warning=error and enables nearly all warnings.


    • I encourage using templates, but with constexpr and concepts metaprogramming is less and less required.


    • "confused Arduino programmers" I would very much like to replace the Arduino (wiring, replication of code in libraries) programming style with a modern C++ approach, which can be easiere, more secure, and produce faster and smaller code. If only I had the time and power....







    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited Aug 28 at 15:19

























    answered Aug 15 at 14:46









    Wouter van Ooijen

    43.3k150114




    43.3k150114











    • Thanks for this answer! Using C++ is an option, but we are using C in my company (what I havent mentioned explicitly). I have updated the question to let people know Im talking about C.
      – L. Heinrichs
      Aug 15 at 14:51










    • Why are you using (only) C? Maybe this gives you chance to convince them to at least consider C++... What you want is essentialy (a small part of) C++ realised in C.
      – Wouter van Ooijen
      Aug 15 at 14:55











    • What I do in my (first 'real' embedded hobby project) is initializing simple default in the constructor, and use a separate Init method for relevant classes. Another benefit is that I can pass stub pointers for unit testing.
      – Michel Keijzers
      Aug 15 at 15:00






    • 2




      @Michel for a hobby project you are free to choose the language? Take C++!
      – Wouter van Ooijen
      Aug 15 at 15:02






    • 4




      And while it is indeed perfectly possible to write good C++ programs for embedded, the problem is that some >50% of all embedded systems programmers out there are quacks, confused PC programmers, Arduino hobbyists etc etc. These kind of people simply cannot use a clean subset of C++, even if you explain it to their face. Give them C++ and before you know it, they will use the whole of STL, template metaprogramming, exception handling, multiple inheritance and so on. And the result is of course complete trash. This is, sadly, how some 8 out of 10 embedded C++ projects end up.
      – Lundin
      Aug 16 at 15:09

















    • Thanks for this answer! Using C++ is an option, but we are using C in my company (what I havent mentioned explicitly). I have updated the question to let people know Im talking about C.
      – L. Heinrichs
      Aug 15 at 14:51










    • Why are you using (only) C? Maybe this gives you chance to convince them to at least consider C++... What you want is essentialy (a small part of) C++ realised in C.
      – Wouter van Ooijen
      Aug 15 at 14:55











    • What I do in my (first 'real' embedded hobby project) is initializing simple default in the constructor, and use a separate Init method for relevant classes. Another benefit is that I can pass stub pointers for unit testing.
      – Michel Keijzers
      Aug 15 at 15:00






    • 2




      @Michel for a hobby project you are free to choose the language? Take C++!
      – Wouter van Ooijen
      Aug 15 at 15:02






    • 4




      And while it is indeed perfectly possible to write good C++ programs for embedded, the problem is that some >50% of all embedded systems programmers out there are quacks, confused PC programmers, Arduino hobbyists etc etc. These kind of people simply cannot use a clean subset of C++, even if you explain it to their face. Give them C++ and before you know it, they will use the whole of STL, template metaprogramming, exception handling, multiple inheritance and so on. And the result is of course complete trash. This is, sadly, how some 8 out of 10 embedded C++ projects end up.
      – Lundin
      Aug 16 at 15:09
















    Thanks for this answer! Using C++ is an option, but we are using C in my company (what I havent mentioned explicitly). I have updated the question to let people know Im talking about C.
    – L. Heinrichs
    Aug 15 at 14:51




    Thanks for this answer! Using C++ is an option, but we are using C in my company (what I havent mentioned explicitly). I have updated the question to let people know Im talking about C.
    – L. Heinrichs
    Aug 15 at 14:51












    Why are you using (only) C? Maybe this gives you chance to convince them to at least consider C++... What you want is essentialy (a small part of) C++ realised in C.
    – Wouter van Ooijen
    Aug 15 at 14:55





    Why are you using (only) C? Maybe this gives you chance to convince them to at least consider C++... What you want is essentialy (a small part of) C++ realised in C.
    – Wouter van Ooijen
    Aug 15 at 14:55













    What I do in my (first 'real' embedded hobby project) is initializing simple default in the constructor, and use a separate Init method for relevant classes. Another benefit is that I can pass stub pointers for unit testing.
    – Michel Keijzers
    Aug 15 at 15:00




    What I do in my (first 'real' embedded hobby project) is initializing simple default in the constructor, and use a separate Init method for relevant classes. Another benefit is that I can pass stub pointers for unit testing.
    – Michel Keijzers
    Aug 15 at 15:00




    2




    2




    @Michel for a hobby project you are free to choose the language? Take C++!
    – Wouter van Ooijen
    Aug 15 at 15:02




    @Michel for a hobby project you are free to choose the language? Take C++!
    – Wouter van Ooijen
    Aug 15 at 15:02




    4




    4




    And while it is indeed perfectly possible to write good C++ programs for embedded, the problem is that some >50% of all embedded systems programmers out there are quacks, confused PC programmers, Arduino hobbyists etc etc. These kind of people simply cannot use a clean subset of C++, even if you explain it to their face. Give them C++ and before you know it, they will use the whole of STL, template metaprogramming, exception handling, multiple inheritance and so on. And the result is of course complete trash. This is, sadly, how some 8 out of 10 embedded C++ projects end up.
    – Lundin
    Aug 16 at 15:09





    And while it is indeed perfectly possible to write good C++ programs for embedded, the problem is that some >50% of all embedded systems programmers out there are quacks, confused PC programmers, Arduino hobbyists etc etc. These kind of people simply cannot use a clean subset of C++, even if you explain it to their face. Give them C++ and before you know it, they will use the whole of STL, template metaprogramming, exception handling, multiple inheritance and so on. And the result is of course complete trash. This is, sadly, how some 8 out of 10 embedded C++ projects end up.
    – Lundin
    Aug 16 at 15:09











    up vote
    7
    down vote













    I believe FreeRTOS (maybe another OS?) does something like what you're looking for by defining 2 different versions of the struct.

    The 'real' one, used internally by the OS functions, and a 'fake' one which is the same size as the 'real' one, but doesn't have any useful members inside (just a bunch of int dummy1 and similar).

    Only the 'fake' struct is exposed outside of the OS code, and this is used to allocate memory to static instances of the struct.

    Internally, when functions in the OS are called, they are passed the address of the external 'fake' struct as a handle, and this is then typecast as a pointer to a 'real' struct so the OS functions can do what they need to do.






    share|improve this answer






















    • Good idea, I guess i could use --- #define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)])) --- BUILD_BUG_ON(sizeof(real_struct) != sizeof(fake_struct)) ----
      – L. Heinrichs
      Aug 15 at 14:47














    up vote
    7
    down vote













    I believe FreeRTOS (maybe another OS?) does something like what you're looking for by defining 2 different versions of the struct.

    The 'real' one, used internally by the OS functions, and a 'fake' one which is the same size as the 'real' one, but doesn't have any useful members inside (just a bunch of int dummy1 and similar).

    Only the 'fake' struct is exposed outside of the OS code, and this is used to allocate memory to static instances of the struct.

    Internally, when functions in the OS are called, they are passed the address of the external 'fake' struct as a handle, and this is then typecast as a pointer to a 'real' struct so the OS functions can do what they need to do.






    share|improve this answer






















    • Good idea, I guess i could use --- #define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)])) --- BUILD_BUG_ON(sizeof(real_struct) != sizeof(fake_struct)) ----
      – L. Heinrichs
      Aug 15 at 14:47












    up vote
    7
    down vote










    up vote
    7
    down vote









    I believe FreeRTOS (maybe another OS?) does something like what you're looking for by defining 2 different versions of the struct.

    The 'real' one, used internally by the OS functions, and a 'fake' one which is the same size as the 'real' one, but doesn't have any useful members inside (just a bunch of int dummy1 and similar).

    Only the 'fake' struct is exposed outside of the OS code, and this is used to allocate memory to static instances of the struct.

    Internally, when functions in the OS are called, they are passed the address of the external 'fake' struct as a handle, and this is then typecast as a pointer to a 'real' struct so the OS functions can do what they need to do.






    share|improve this answer














    I believe FreeRTOS (maybe another OS?) does something like what you're looking for by defining 2 different versions of the struct.

    The 'real' one, used internally by the OS functions, and a 'fake' one which is the same size as the 'real' one, but doesn't have any useful members inside (just a bunch of int dummy1 and similar).

    Only the 'fake' struct is exposed outside of the OS code, and this is used to allocate memory to static instances of the struct.

    Internally, when functions in the OS are called, they are passed the address of the external 'fake' struct as a handle, and this is then typecast as a pointer to a 'real' struct so the OS functions can do what they need to do.







    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited Aug 15 at 14:42

























    answered Aug 15 at 14:31









    brhans

    8,35221726




    8,35221726











    • Good idea, I guess i could use --- #define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)])) --- BUILD_BUG_ON(sizeof(real_struct) != sizeof(fake_struct)) ----
      – L. Heinrichs
      Aug 15 at 14:47
















    • Good idea, I guess i could use --- #define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)])) --- BUILD_BUG_ON(sizeof(real_struct) != sizeof(fake_struct)) ----
      – L. Heinrichs
      Aug 15 at 14:47















    Good idea, I guess i could use --- #define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)])) --- BUILD_BUG_ON(sizeof(real_struct) != sizeof(fake_struct)) ----
    – L. Heinrichs
    Aug 15 at 14:47




    Good idea, I guess i could use --- #define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)])) --- BUILD_BUG_ON(sizeof(real_struct) != sizeof(fake_struct)) ----
    – L. Heinrichs
    Aug 15 at 14:47










    up vote
    2
    down vote














    Anonymous struct, so the struct may only be altered by the use of provided interface functions




    In my opinion, this is pointless. You can put a comment there, but no point trying to hide it further.



    C will never provide such high isolation, even if there is no declaration for the struct, it will be easy to accidentally overwrite it with e.g. mistaken memcpy() or buffer overflow.



    Instead, just give the struct a name and trust other people to write good code also. It will also make debugging easier when the struct has a name you can use to refer to it.






    share|improve this answer
























      up vote
      2
      down vote














      Anonymous struct, so the struct may only be altered by the use of provided interface functions




      In my opinion, this is pointless. You can put a comment there, but no point trying to hide it further.



      C will never provide such high isolation, even if there is no declaration for the struct, it will be easy to accidentally overwrite it with e.g. mistaken memcpy() or buffer overflow.



      Instead, just give the struct a name and trust other people to write good code also. It will also make debugging easier when the struct has a name you can use to refer to it.






      share|improve this answer






















        up vote
        2
        down vote










        up vote
        2
        down vote










        Anonymous struct, so the struct may only be altered by the use of provided interface functions




        In my opinion, this is pointless. You can put a comment there, but no point trying to hide it further.



        C will never provide such high isolation, even if there is no declaration for the struct, it will be easy to accidentally overwrite it with e.g. mistaken memcpy() or buffer overflow.



        Instead, just give the struct a name and trust other people to write good code also. It will also make debugging easier when the struct has a name you can use to refer to it.






        share|improve this answer













        Anonymous struct, so the struct may only be altered by the use of provided interface functions




        In my opinion, this is pointless. You can put a comment there, but no point trying to hide it further.



        C will never provide such high isolation, even if there is no declaration for the struct, it will be easy to accidentally overwrite it with e.g. mistaken memcpy() or buffer overflow.



        Instead, just give the struct a name and trust other people to write good code also. It will also make debugging easier when the struct has a name you can use to refer to it.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Aug 16 at 10:15









        jpa

        1,265611




        1,265611




















            up vote
            2
            down vote













            Pure SW questions are better asked at https://stackoverflow.com.



            The concept with exposing a struct of incomplete type to caller, like you describe, is often called "opaque type" or "opaque pointers" - anonymous struct means something else entirely.



            The problem with this is that the caller won't be able to allocate instances of the object, only pointers to it. On a PC you would use malloc inside the objects "constructor", but malloc is a no-go in embedded systems.



            So what you do in embedded is to provide a memory pool. You have a limited amount of RAM, so restricting the number of objects that can be created is usually not a problem.



            See Static allocation of opaque data types over at SO.






            share|improve this answer




















            • Ou thank you for clarifying the naming confusion on my end, ill adjust the OP. I was thinking of going to stack overflow, but decided i would like to target embedded programmers specifically.
              – L. Heinrichs
              Aug 17 at 17:04














            up vote
            2
            down vote













            Pure SW questions are better asked at https://stackoverflow.com.



            The concept with exposing a struct of incomplete type to caller, like you describe, is often called "opaque type" or "opaque pointers" - anonymous struct means something else entirely.



            The problem with this is that the caller won't be able to allocate instances of the object, only pointers to it. On a PC you would use malloc inside the objects "constructor", but malloc is a no-go in embedded systems.



            So what you do in embedded is to provide a memory pool. You have a limited amount of RAM, so restricting the number of objects that can be created is usually not a problem.



            See Static allocation of opaque data types over at SO.






            share|improve this answer




















            • Ou thank you for clarifying the naming confusion on my end, ill adjust the OP. I was thinking of going to stack overflow, but decided i would like to target embedded programmers specifically.
              – L. Heinrichs
              Aug 17 at 17:04












            up vote
            2
            down vote










            up vote
            2
            down vote









            Pure SW questions are better asked at https://stackoverflow.com.



            The concept with exposing a struct of incomplete type to caller, like you describe, is often called "opaque type" or "opaque pointers" - anonymous struct means something else entirely.



            The problem with this is that the caller won't be able to allocate instances of the object, only pointers to it. On a PC you would use malloc inside the objects "constructor", but malloc is a no-go in embedded systems.



            So what you do in embedded is to provide a memory pool. You have a limited amount of RAM, so restricting the number of objects that can be created is usually not a problem.



            See Static allocation of opaque data types over at SO.






            share|improve this answer












            Pure SW questions are better asked at https://stackoverflow.com.



            The concept with exposing a struct of incomplete type to caller, like you describe, is often called "opaque type" or "opaque pointers" - anonymous struct means something else entirely.



            The problem with this is that the caller won't be able to allocate instances of the object, only pointers to it. On a PC you would use malloc inside the objects "constructor", but malloc is a no-go in embedded systems.



            So what you do in embedded is to provide a memory pool. You have a limited amount of RAM, so restricting the number of objects that can be created is usually not a problem.



            See Static allocation of opaque data types over at SO.







            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Aug 16 at 15:02









            Lundin

            3,301929




            3,301929











            • Ou thank you for clarifying the naming confusion on my end, ill adjust the OP. I was thinking of going to stack overflow, but decided i would like to target embedded programmers specifically.
              – L. Heinrichs
              Aug 17 at 17:04
















            • Ou thank you for clarifying the naming confusion on my end, ill adjust the OP. I was thinking of going to stack overflow, but decided i would like to target embedded programmers specifically.
              – L. Heinrichs
              Aug 17 at 17:04















            Ou thank you for clarifying the naming confusion on my end, ill adjust the OP. I was thinking of going to stack overflow, but decided i would like to target embedded programmers specifically.
            – L. Heinrichs
            Aug 17 at 17:04




            Ou thank you for clarifying the naming confusion on my end, ill adjust the OP. I was thinking of going to stack overflow, but decided i would like to target embedded programmers specifically.
            – L. Heinrichs
            Aug 17 at 17:04

















             

            draft saved


            draft discarded















































             


            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2felectronics.stackexchange.com%2fquestions%2f391178%2fpossibilities-for-allocating-memory-for-modular-firmware-design-in-c%23new-answer', 'question_page');

            );

            Post as a guest













































































            Comments

            Popular posts from this blog

            What does second last employer means? [closed]

            List of Gilmore Girls characters

            Confectionery