Possibilities for allocating memory for modular firmware design in C
Clash Royale CLAN TAG#URR8PPP
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0;
up vote
16
down vote
favorite
modular approaches are pretty handy in general (portable and clean), so I try to program modules as independent of any other modules as possible. Most of my approaches are based on a struct that describes the module itself. An initialization function sets the primary parameters, afterwards a handler (pointer to desriptive struct) is passed to whatever function inside the module is called.
Right now, I am wondering what the best approach of allocation memory for the struct describing a module may be. If possible, I'd like the following:
- Opaque struct, so the struct may only be altered by the use of provided interface functions
- Multiple instances
- memory allocated by linker
I see the following possibilities, that all conflict with one of my goals:
global declaration
multiple instances, allcoted by linker, but struct is not opaque
(#includes)
module_struct module;
void main()
module_init(&module);
malloc
opaque struct, multiple instances, but allcotion on heap
in module.h:
typedef module_struct Module;
in module.c init function, malloc and return pointer to allocated memory
module_mem = malloc(sizeof(module_struct ));
/* initialize values here */
return module_mem;
in main.c
(#includes)
Module *module;
void main()
module = module_init();
declaration in module
opaque struct, allocated by linker, only a predefined number of instances
keep the whole struct and memory internal to the module and never expose a handler or struct.
(#includes)
void main()
module_init(_no_param_or_index_if_multiple_instances_possible_);
Is there an option to combine these somehow for opaque struct, linker instead of heap allocation and multiple/any number of instances?
solution
as proposed in some answers below, I think the best way is to:
- reserve space for MODULE_MAX_INSTANCE_COUNT modules in the modules source file
- do not define MODULE_MAX_INSTANCE_COUNT in the module itsself
- add an #ifndef MODULE_MAX_INSTANCE_COUNT #error to the modules header file to make sure the modules user is aware of this limitation and defines the maximum number of instances wanted for the application
- on initialization of an instance return either the memory address (*void) of the desricptive struct or the modules index (whatever you like more)
c design firmware
 |Â
show 5 more comments
up vote
16
down vote
favorite
modular approaches are pretty handy in general (portable and clean), so I try to program modules as independent of any other modules as possible. Most of my approaches are based on a struct that describes the module itself. An initialization function sets the primary parameters, afterwards a handler (pointer to desriptive struct) is passed to whatever function inside the module is called.
Right now, I am wondering what the best approach of allocation memory for the struct describing a module may be. If possible, I'd like the following:
- Opaque struct, so the struct may only be altered by the use of provided interface functions
- Multiple instances
- memory allocated by linker
I see the following possibilities, that all conflict with one of my goals:
global declaration
multiple instances, allcoted by linker, but struct is not opaque
(#includes)
module_struct module;
void main()
module_init(&module);
malloc
opaque struct, multiple instances, but allcotion on heap
in module.h:
typedef module_struct Module;
in module.c init function, malloc and return pointer to allocated memory
module_mem = malloc(sizeof(module_struct ));
/* initialize values here */
return module_mem;
in main.c
(#includes)
Module *module;
void main()
module = module_init();
declaration in module
opaque struct, allocated by linker, only a predefined number of instances
keep the whole struct and memory internal to the module and never expose a handler or struct.
(#includes)
void main()
module_init(_no_param_or_index_if_multiple_instances_possible_);
Is there an option to combine these somehow for opaque struct, linker instead of heap allocation and multiple/any number of instances?
solution
as proposed in some answers below, I think the best way is to:
- reserve space for MODULE_MAX_INSTANCE_COUNT modules in the modules source file
- do not define MODULE_MAX_INSTANCE_COUNT in the module itsself
- add an #ifndef MODULE_MAX_INSTANCE_COUNT #error to the modules header file to make sure the modules user is aware of this limitation and defines the maximum number of instances wanted for the application
- on initialization of an instance return either the memory address (*void) of the desricptive struct or the modules index (whatever you like more)
c design firmware
12
Most embedded FW designers are avoiding the dynamic allocation to keep the memory usage deterministic and simple. Especially if it is bare-metal and does not have underlying OS to manage the memory.
– Eugene Sh.
Aug 15 at 14:25
Exactly, thats why I want the linker to do the allocations.
– L. Heinrichs
Aug 15 at 14:28
4
I'm not quite sure I understand... How could you have the memory allocated by the linker if you have a dynamic number of instances? That seems quite orthogonal to me.
– jcaron
Aug 15 at 17:13
Why not let the linker allocate one big memory pool, and do your own allocations from that, which also gives you the benefit of a zero-overhead allocator. You can make the pool object static to the file with the allocation function so it's private. In some of my code, I do all the allocations in the various init routines, then I print out afterward how much was allocated, so in the final production compile I set the pool to that exact size.
– Lee Daniel Crocker
Aug 15 at 23:58
2
If it's a compile-time decision, you could simply define the number in your Makefile or equivalent, and you're all set. The number would not be in the source of the module, but would be application specific, and you just use an instance number as a parameter.
– jcaron
Aug 16 at 14:49
 |Â
show 5 more comments
up vote
16
down vote
favorite
up vote
16
down vote
favorite
modular approaches are pretty handy in general (portable and clean), so I try to program modules as independent of any other modules as possible. Most of my approaches are based on a struct that describes the module itself. An initialization function sets the primary parameters, afterwards a handler (pointer to desriptive struct) is passed to whatever function inside the module is called.
Right now, I am wondering what the best approach of allocation memory for the struct describing a module may be. If possible, I'd like the following:
- Opaque struct, so the struct may only be altered by the use of provided interface functions
- Multiple instances
- memory allocated by linker
I see the following possibilities, that all conflict with one of my goals:
global declaration
multiple instances, allcoted by linker, but struct is not opaque
(#includes)
module_struct module;
void main()
module_init(&module);
malloc
opaque struct, multiple instances, but allcotion on heap
in module.h:
typedef module_struct Module;
in module.c init function, malloc and return pointer to allocated memory
module_mem = malloc(sizeof(module_struct ));
/* initialize values here */
return module_mem;
in main.c
(#includes)
Module *module;
void main()
module = module_init();
declaration in module
opaque struct, allocated by linker, only a predefined number of instances
keep the whole struct and memory internal to the module and never expose a handler or struct.
(#includes)
void main()
module_init(_no_param_or_index_if_multiple_instances_possible_);
Is there an option to combine these somehow for opaque struct, linker instead of heap allocation and multiple/any number of instances?
solution
as proposed in some answers below, I think the best way is to:
- reserve space for MODULE_MAX_INSTANCE_COUNT modules in the modules source file
- do not define MODULE_MAX_INSTANCE_COUNT in the module itsself
- add an #ifndef MODULE_MAX_INSTANCE_COUNT #error to the modules header file to make sure the modules user is aware of this limitation and defines the maximum number of instances wanted for the application
- on initialization of an instance return either the memory address (*void) of the desricptive struct or the modules index (whatever you like more)
c design firmware
modular approaches are pretty handy in general (portable and clean), so I try to program modules as independent of any other modules as possible. Most of my approaches are based on a struct that describes the module itself. An initialization function sets the primary parameters, afterwards a handler (pointer to desriptive struct) is passed to whatever function inside the module is called.
Right now, I am wondering what the best approach of allocation memory for the struct describing a module may be. If possible, I'd like the following:
- Opaque struct, so the struct may only be altered by the use of provided interface functions
- Multiple instances
- memory allocated by linker
I see the following possibilities, that all conflict with one of my goals:
global declaration
multiple instances, allcoted by linker, but struct is not opaque
(#includes)
module_struct module;
void main()
module_init(&module);
malloc
opaque struct, multiple instances, but allcotion on heap
in module.h:
typedef module_struct Module;
in module.c init function, malloc and return pointer to allocated memory
module_mem = malloc(sizeof(module_struct ));
/* initialize values here */
return module_mem;
in main.c
(#includes)
Module *module;
void main()
module = module_init();
declaration in module
opaque struct, allocated by linker, only a predefined number of instances
keep the whole struct and memory internal to the module and never expose a handler or struct.
(#includes)
void main()
module_init(_no_param_or_index_if_multiple_instances_possible_);
Is there an option to combine these somehow for opaque struct, linker instead of heap allocation and multiple/any number of instances?
solution
as proposed in some answers below, I think the best way is to:
- reserve space for MODULE_MAX_INSTANCE_COUNT modules in the modules source file
- do not define MODULE_MAX_INSTANCE_COUNT in the module itsself
- add an #ifndef MODULE_MAX_INSTANCE_COUNT #error to the modules header file to make sure the modules user is aware of this limitation and defines the maximum number of instances wanted for the application
- on initialization of an instance return either the memory address (*void) of the desricptive struct or the modules index (whatever you like more)
c design firmware
edited Sep 1 at 10:15
asked Aug 15 at 14:22
L. Heinrichs
343111
343111
12
Most embedded FW designers are avoiding the dynamic allocation to keep the memory usage deterministic and simple. Especially if it is bare-metal and does not have underlying OS to manage the memory.
– Eugene Sh.
Aug 15 at 14:25
Exactly, thats why I want the linker to do the allocations.
– L. Heinrichs
Aug 15 at 14:28
4
I'm not quite sure I understand... How could you have the memory allocated by the linker if you have a dynamic number of instances? That seems quite orthogonal to me.
– jcaron
Aug 15 at 17:13
Why not let the linker allocate one big memory pool, and do your own allocations from that, which also gives you the benefit of a zero-overhead allocator. You can make the pool object static to the file with the allocation function so it's private. In some of my code, I do all the allocations in the various init routines, then I print out afterward how much was allocated, so in the final production compile I set the pool to that exact size.
– Lee Daniel Crocker
Aug 15 at 23:58
2
If it's a compile-time decision, you could simply define the number in your Makefile or equivalent, and you're all set. The number would not be in the source of the module, but would be application specific, and you just use an instance number as a parameter.
– jcaron
Aug 16 at 14:49
 |Â
show 5 more comments
12
Most embedded FW designers are avoiding the dynamic allocation to keep the memory usage deterministic and simple. Especially if it is bare-metal and does not have underlying OS to manage the memory.
– Eugene Sh.
Aug 15 at 14:25
Exactly, thats why I want the linker to do the allocations.
– L. Heinrichs
Aug 15 at 14:28
4
I'm not quite sure I understand... How could you have the memory allocated by the linker if you have a dynamic number of instances? That seems quite orthogonal to me.
– jcaron
Aug 15 at 17:13
Why not let the linker allocate one big memory pool, and do your own allocations from that, which also gives you the benefit of a zero-overhead allocator. You can make the pool object static to the file with the allocation function so it's private. In some of my code, I do all the allocations in the various init routines, then I print out afterward how much was allocated, so in the final production compile I set the pool to that exact size.
– Lee Daniel Crocker
Aug 15 at 23:58
2
If it's a compile-time decision, you could simply define the number in your Makefile or equivalent, and you're all set. The number would not be in the source of the module, but would be application specific, and you just use an instance number as a parameter.
– jcaron
Aug 16 at 14:49
12
12
Most embedded FW designers are avoiding the dynamic allocation to keep the memory usage deterministic and simple. Especially if it is bare-metal and does not have underlying OS to manage the memory.
– Eugene Sh.
Aug 15 at 14:25
Most embedded FW designers are avoiding the dynamic allocation to keep the memory usage deterministic and simple. Especially if it is bare-metal and does not have underlying OS to manage the memory.
– Eugene Sh.
Aug 15 at 14:25
Exactly, thats why I want the linker to do the allocations.
– L. Heinrichs
Aug 15 at 14:28
Exactly, thats why I want the linker to do the allocations.
– L. Heinrichs
Aug 15 at 14:28
4
4
I'm not quite sure I understand... How could you have the memory allocated by the linker if you have a dynamic number of instances? That seems quite orthogonal to me.
– jcaron
Aug 15 at 17:13
I'm not quite sure I understand... How could you have the memory allocated by the linker if you have a dynamic number of instances? That seems quite orthogonal to me.
– jcaron
Aug 15 at 17:13
Why not let the linker allocate one big memory pool, and do your own allocations from that, which also gives you the benefit of a zero-overhead allocator. You can make the pool object static to the file with the allocation function so it's private. In some of my code, I do all the allocations in the various init routines, then I print out afterward how much was allocated, so in the final production compile I set the pool to that exact size.
– Lee Daniel Crocker
Aug 15 at 23:58
Why not let the linker allocate one big memory pool, and do your own allocations from that, which also gives you the benefit of a zero-overhead allocator. You can make the pool object static to the file with the allocation function so it's private. In some of my code, I do all the allocations in the various init routines, then I print out afterward how much was allocated, so in the final production compile I set the pool to that exact size.
– Lee Daniel Crocker
Aug 15 at 23:58
2
2
If it's a compile-time decision, you could simply define the number in your Makefile or equivalent, and you're all set. The number would not be in the source of the module, but would be application specific, and you just use an instance number as a parameter.
– jcaron
Aug 16 at 14:49
If it's a compile-time decision, you could simply define the number in your Makefile or equivalent, and you're all set. The number would not be in the source of the module, but would be application specific, and you just use an instance number as a parameter.
– jcaron
Aug 16 at 14:49
 |Â
show 5 more comments
5 Answers
5
active
oldest
votes
up vote
3
down vote
accepted
Is there an option to combine these somehow for anonymous struct,
linker instead of heap allocation and multiple/any number of
instances?
Sure there is. First, however, recognize that the "any number" of instances must be fixed, or at least an upper bound established, at compile time. This is a prerequisite for the instances to be statically allocated (what you're calling "linker allocation"). You might make the number adjustable without source modification by declaring a macro that specifies it.
Then the source file containing the actual struct declaration and all its associated functions also declares an array of instances with internal linkage. It provides either an array, with external linkage, of pointers to the instances or else a function for accessing the various pointers by index. The function variation is a bit more modular:
module.c
#include <module.h>
// 4 instances by default; can be overridden at compile time
#ifndef NUM_MODULE_INSTANCES
#define NUM_MODULE_INSTANCES 4
#endif
struct module
int demo;
;
// has internal linkage, so is not directly visible from other files:
static struct module instances[NUM_MODULE_INSTANCES];
// module functions
struct module *module_init(unsigned index)
instances[index].demo = 42;
return &instances[index];
I guess you're already familiar with how the header would then declare the struct as an incomplete type and declare all the functions (written in terms of pointers to that type). For example:
module.h
#ifndef MODULE_H
#define MODULE_H
struct module;
struct module *module_init(unsigned index);
// other functions ...
#endif
Now struct module
is opaque in translation units other than module.c
,* and you can access and use up to the number of instances defined at compile time without any dynamic allocation.
*Unless you copy its definition, of course. The point is that module.h
does not do that.
I think it's an odd design to pass the index from outside the class. When I implement memory pools like this, I let the index be a private counter, increasing by 1 for every allocated instance. Until you reach "NUM_MODULE_INSTANCES", where the constructor will return an out of memory error.
– Lundin
Aug 17 at 6:16
That's a fair point, @Lundin. That aspect of the design supposes that the indexes have inherent significance, which may or may not in fact be the case. It is the case, albeit trivially so, for the OP's starting case. Such significance, if it exists, could be further supported by providing a means to obtain an instance pointer without initializing.
– John Bollinger
Aug 17 at 11:51
So basically you reserve memory for n modules, no matter how many will be used, then return a pointer to the next unused element if the application initializes it. I guess that could work.
– L. Heinrichs
Aug 17 at 17:05
@L.Heinrichs Yes, since embedded systems are of a deterministic nature. There's no such thing as "endless amount of objects" nor "unknown amount of objects". Objects are often singletons too (hardware drivers), so there's often no need for the memory pool, as just one single instance of the object will exist.
– Lundin
Aug 20 at 14:06
I agree for most cases. The question had some theoretical interest as well. But I could use hundreds of 1-wire temperature sensors if there are enough IOs available (as the one example I can come up with now).
– L. Heinrichs
Aug 20 at 15:08
add a comment |Â
up vote
22
down vote
I program small micro-controllers in C++, which achieves exactly what you want.
What you call a module is a C++ class, it can contain data (either externally accessible or not) and functions (likewise). The constructor (a dedicated function) initilializes it. The constructor can take run-time parameters or (my favourite) compile-time (template) parameters. The functions within the class implicitly get the class variable as first parameter. (Or, often my preference, the class can act as a hidden singleton, so all data is accessed without this overhead).
The class object can be global (so you know at link-time that everything will fit), or stack-local, presumably in the main. (I don't like C++ globals because of the undefined global initialization order, so I prefer stack-local).
My preferred programming style is that modules are static classes, and their (static) configuration is by template parameters. This avoids nearly all overhad and enables optimization. Combine this with a tool that calculates the stack size and you can sleep without worries :)
My talk on this way of coding in C++: Objects? No Thanks!
A lot of embedded / microcontroller programmers seem to dislike C++ because they think it would force them to use all of C++. That is absolutely not neccessary, and would be a very bad idea. (You probably don't use all of C either! Think heap, floating point, setjmp/longjmp, printf, ...)
In a comment Adam Haun mentions RAII and initialization. IMO RAII has more to do with deconstruction, but his point is valid: global objects will be constructed before your main starts, so they might work on invalid assumptions (like a main clock speed that will be changed lateron). That is one more reason NOT to use global code-initialized objects. (I use a linker script that will fail when I have global code-initialized objects.) IMO such 'objects' should be explicitly created and passed around. This includes a 'waiting' facility 'object' that provides a wait() function. In my setup this is 'object' that sets the clock speed of the chip.
Talking about RAII: that is one more C++ feature that is very usefull in small embedded systems, although not for the reason (memory deallocation) it most used for in largere systems (small embedded systems mostly don't use dynamic memory deallocation). Think of locking a resource: you can make the locked resource a wrapper object, and restrict access to the resource to be only possible via the locking wrapper. When the wrapper goes out of scope, the resource is unlocked. This prevents access without locking, and makes it much more unlikely to forget the unlocking. with some (template) magic it can be zero-overhead.
The original question didn't mention C, hence my C++-centric answer. If it realy must be C....
You could use macro trickery: declare your stucts publicly, so they have a type and can be allocated globally, but mangle the names of their components beyond usability, unless some macro is defined differently, which is the case in your module's .c file. For extra security you could use the compile time in the mangling.
Or have a public version of your struct that has nothing usefull in it, and have the private version (with usefull data) only in your .c file, and assert that they are the same size. A bit of make-file trickery could automate this.
@Lundins comment about bad (embedded) programmers:
The type of programmer you describe would probably make a mess in any language. Macro's (present in C and C++) are one obvious way.
Tooling can help to some extent. For my students I mandate a built script that specifies no-exceptions, no-rtti, and gives a linker error when either the heap is used or code-initialized globals are present. And it specifies warning=error and enables nearly all warnings.
I encourage using templates, but with constexpr and concepts metaprogramming is less and less required.
"confused Arduino programmers" I would very much like to replace the Arduino (wiring, replication of code in libraries) programming style with a modern C++ approach, which can be easiere, more secure, and produce faster and smaller code. If only I had the time and power....
Thanks for this answer! Using C++ is an option, but we are using C in my company (what I havent mentioned explicitly). I have updated the question to let people know Im talking about C.
– L. Heinrichs
Aug 15 at 14:51
Why are you using (only) C? Maybe this gives you chance to convince them to at least consider C++... What you want is essentialy (a small part of) C++ realised in C.
– Wouter van Ooijen
Aug 15 at 14:55
What I do in my (first 'real' embedded hobby project) is initializing simple default in the constructor, and use a separate Init method for relevant classes. Another benefit is that I can pass stub pointers for unit testing.
– Michel Keijzers
Aug 15 at 15:00
2
@Michel for a hobby project you are free to choose the language? Take C++!
– Wouter van Ooijen
Aug 15 at 15:02
4
And while it is indeed perfectly possible to write good C++ programs for embedded, the problem is that some >50% of all embedded systems programmers out there are quacks, confused PC programmers, Arduino hobbyists etc etc. These kind of people simply cannot use a clean subset of C++, even if you explain it to their face. Give them C++ and before you know it, they will use the whole of STL, template metaprogramming, exception handling, multiple inheritance and so on. And the result is of course complete trash. This is, sadly, how some 8 out of 10 embedded C++ projects end up.
– Lundin
Aug 16 at 15:09
 |Â
show 9 more comments
up vote
7
down vote
I believe FreeRTOS (maybe another OS?) does something like what you're looking for by defining 2 different versions of the struct.
The 'real' one, used internally by the OS functions, and a 'fake' one which is the same size as the 'real' one, but doesn't have any useful members inside (just a bunch of int dummy1
and similar).
Only the 'fake' struct is exposed outside of the OS code, and this is used to allocate memory to static instances of the struct.
Internally, when functions in the OS are called, they are passed the address of the external 'fake' struct as a handle, and this is then typecast as a pointer to a 'real' struct so the OS functions can do what they need to do.
Good idea, I guess i could use --- #define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)])) --- BUILD_BUG_ON(sizeof(real_struct) != sizeof(fake_struct)) ----
– L. Heinrichs
Aug 15 at 14:47
add a comment |Â
up vote
2
down vote
Anonymous struct, so the struct may only be altered by the use of provided interface functions
In my opinion, this is pointless. You can put a comment there, but no point trying to hide it further.
C will never provide such high isolation, even if there is no declaration for the struct, it will be easy to accidentally overwrite it with e.g. mistaken memcpy() or buffer overflow.
Instead, just give the struct a name and trust other people to write good code also. It will also make debugging easier when the struct has a name you can use to refer to it.
add a comment |Â
up vote
2
down vote
Pure SW questions are better asked at https://stackoverflow.com.
The concept with exposing a struct of incomplete type to caller, like you describe, is often called "opaque type" or "opaque pointers" - anonymous struct means something else entirely.
The problem with this is that the caller won't be able to allocate instances of the object, only pointers to it. On a PC you would use malloc
inside the objects "constructor", but malloc is a no-go in embedded systems.
So what you do in embedded is to provide a memory pool. You have a limited amount of RAM, so restricting the number of objects that can be created is usually not a problem.
See Static allocation of opaque data types over at SO.
Ou thank you for clarifying the naming confusion on my end, ill adjust the OP. I was thinking of going to stack overflow, but decided i would like to target embedded programmers specifically.
– L. Heinrichs
Aug 17 at 17:04
add a comment |Â
5 Answers
5
active
oldest
votes
5 Answers
5
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
3
down vote
accepted
Is there an option to combine these somehow for anonymous struct,
linker instead of heap allocation and multiple/any number of
instances?
Sure there is. First, however, recognize that the "any number" of instances must be fixed, or at least an upper bound established, at compile time. This is a prerequisite for the instances to be statically allocated (what you're calling "linker allocation"). You might make the number adjustable without source modification by declaring a macro that specifies it.
Then the source file containing the actual struct declaration and all its associated functions also declares an array of instances with internal linkage. It provides either an array, with external linkage, of pointers to the instances or else a function for accessing the various pointers by index. The function variation is a bit more modular:
module.c
#include <module.h>
// 4 instances by default; can be overridden at compile time
#ifndef NUM_MODULE_INSTANCES
#define NUM_MODULE_INSTANCES 4
#endif
struct module
int demo;
;
// has internal linkage, so is not directly visible from other files:
static struct module instances[NUM_MODULE_INSTANCES];
// module functions
struct module *module_init(unsigned index)
instances[index].demo = 42;
return &instances[index];
I guess you're already familiar with how the header would then declare the struct as an incomplete type and declare all the functions (written in terms of pointers to that type). For example:
module.h
#ifndef MODULE_H
#define MODULE_H
struct module;
struct module *module_init(unsigned index);
// other functions ...
#endif
Now struct module
is opaque in translation units other than module.c
,* and you can access and use up to the number of instances defined at compile time without any dynamic allocation.
*Unless you copy its definition, of course. The point is that module.h
does not do that.
I think it's an odd design to pass the index from outside the class. When I implement memory pools like this, I let the index be a private counter, increasing by 1 for every allocated instance. Until you reach "NUM_MODULE_INSTANCES", where the constructor will return an out of memory error.
– Lundin
Aug 17 at 6:16
That's a fair point, @Lundin. That aspect of the design supposes that the indexes have inherent significance, which may or may not in fact be the case. It is the case, albeit trivially so, for the OP's starting case. Such significance, if it exists, could be further supported by providing a means to obtain an instance pointer without initializing.
– John Bollinger
Aug 17 at 11:51
So basically you reserve memory for n modules, no matter how many will be used, then return a pointer to the next unused element if the application initializes it. I guess that could work.
– L. Heinrichs
Aug 17 at 17:05
@L.Heinrichs Yes, since embedded systems are of a deterministic nature. There's no such thing as "endless amount of objects" nor "unknown amount of objects". Objects are often singletons too (hardware drivers), so there's often no need for the memory pool, as just one single instance of the object will exist.
– Lundin
Aug 20 at 14:06
I agree for most cases. The question had some theoretical interest as well. But I could use hundreds of 1-wire temperature sensors if there are enough IOs available (as the one example I can come up with now).
– L. Heinrichs
Aug 20 at 15:08
add a comment |Â
up vote
3
down vote
accepted
Is there an option to combine these somehow for anonymous struct,
linker instead of heap allocation and multiple/any number of
instances?
Sure there is. First, however, recognize that the "any number" of instances must be fixed, or at least an upper bound established, at compile time. This is a prerequisite for the instances to be statically allocated (what you're calling "linker allocation"). You might make the number adjustable without source modification by declaring a macro that specifies it.
Then the source file containing the actual struct declaration and all its associated functions also declares an array of instances with internal linkage. It provides either an array, with external linkage, of pointers to the instances or else a function for accessing the various pointers by index. The function variation is a bit more modular:
module.c
#include <module.h>
// 4 instances by default; can be overridden at compile time
#ifndef NUM_MODULE_INSTANCES
#define NUM_MODULE_INSTANCES 4
#endif
struct module
int demo;
;
// has internal linkage, so is not directly visible from other files:
static struct module instances[NUM_MODULE_INSTANCES];
// module functions
struct module *module_init(unsigned index)
instances[index].demo = 42;
return &instances[index];
I guess you're already familiar with how the header would then declare the struct as an incomplete type and declare all the functions (written in terms of pointers to that type). For example:
module.h
#ifndef MODULE_H
#define MODULE_H
struct module;
struct module *module_init(unsigned index);
// other functions ...
#endif
Now struct module
is opaque in translation units other than module.c
,* and you can access and use up to the number of instances defined at compile time without any dynamic allocation.
*Unless you copy its definition, of course. The point is that module.h
does not do that.
I think it's an odd design to pass the index from outside the class. When I implement memory pools like this, I let the index be a private counter, increasing by 1 for every allocated instance. Until you reach "NUM_MODULE_INSTANCES", where the constructor will return an out of memory error.
– Lundin
Aug 17 at 6:16
That's a fair point, @Lundin. That aspect of the design supposes that the indexes have inherent significance, which may or may not in fact be the case. It is the case, albeit trivially so, for the OP's starting case. Such significance, if it exists, could be further supported by providing a means to obtain an instance pointer without initializing.
– John Bollinger
Aug 17 at 11:51
So basically you reserve memory for n modules, no matter how many will be used, then return a pointer to the next unused element if the application initializes it. I guess that could work.
– L. Heinrichs
Aug 17 at 17:05
@L.Heinrichs Yes, since embedded systems are of a deterministic nature. There's no such thing as "endless amount of objects" nor "unknown amount of objects". Objects are often singletons too (hardware drivers), so there's often no need for the memory pool, as just one single instance of the object will exist.
– Lundin
Aug 20 at 14:06
I agree for most cases. The question had some theoretical interest as well. But I could use hundreds of 1-wire temperature sensors if there are enough IOs available (as the one example I can come up with now).
– L. Heinrichs
Aug 20 at 15:08
add a comment |Â
up vote
3
down vote
accepted
up vote
3
down vote
accepted
Is there an option to combine these somehow for anonymous struct,
linker instead of heap allocation and multiple/any number of
instances?
Sure there is. First, however, recognize that the "any number" of instances must be fixed, or at least an upper bound established, at compile time. This is a prerequisite for the instances to be statically allocated (what you're calling "linker allocation"). You might make the number adjustable without source modification by declaring a macro that specifies it.
Then the source file containing the actual struct declaration and all its associated functions also declares an array of instances with internal linkage. It provides either an array, with external linkage, of pointers to the instances or else a function for accessing the various pointers by index. The function variation is a bit more modular:
module.c
#include <module.h>
// 4 instances by default; can be overridden at compile time
#ifndef NUM_MODULE_INSTANCES
#define NUM_MODULE_INSTANCES 4
#endif
struct module
int demo;
;
// has internal linkage, so is not directly visible from other files:
static struct module instances[NUM_MODULE_INSTANCES];
// module functions
struct module *module_init(unsigned index)
instances[index].demo = 42;
return &instances[index];
I guess you're already familiar with how the header would then declare the struct as an incomplete type and declare all the functions (written in terms of pointers to that type). For example:
module.h
#ifndef MODULE_H
#define MODULE_H
struct module;
struct module *module_init(unsigned index);
// other functions ...
#endif
Now struct module
is opaque in translation units other than module.c
,* and you can access and use up to the number of instances defined at compile time without any dynamic allocation.
*Unless you copy its definition, of course. The point is that module.h
does not do that.
Is there an option to combine these somehow for anonymous struct,
linker instead of heap allocation and multiple/any number of
instances?
Sure there is. First, however, recognize that the "any number" of instances must be fixed, or at least an upper bound established, at compile time. This is a prerequisite for the instances to be statically allocated (what you're calling "linker allocation"). You might make the number adjustable without source modification by declaring a macro that specifies it.
Then the source file containing the actual struct declaration and all its associated functions also declares an array of instances with internal linkage. It provides either an array, with external linkage, of pointers to the instances or else a function for accessing the various pointers by index. The function variation is a bit more modular:
module.c
#include <module.h>
// 4 instances by default; can be overridden at compile time
#ifndef NUM_MODULE_INSTANCES
#define NUM_MODULE_INSTANCES 4
#endif
struct module
int demo;
;
// has internal linkage, so is not directly visible from other files:
static struct module instances[NUM_MODULE_INSTANCES];
// module functions
struct module *module_init(unsigned index)
instances[index].demo = 42;
return &instances[index];
I guess you're already familiar with how the header would then declare the struct as an incomplete type and declare all the functions (written in terms of pointers to that type). For example:
module.h
#ifndef MODULE_H
#define MODULE_H
struct module;
struct module *module_init(unsigned index);
// other functions ...
#endif
Now struct module
is opaque in translation units other than module.c
,* and you can access and use up to the number of instances defined at compile time without any dynamic allocation.
*Unless you copy its definition, of course. The point is that module.h
does not do that.
answered Aug 16 at 15:25
John Bollinger
1461
1461
I think it's an odd design to pass the index from outside the class. When I implement memory pools like this, I let the index be a private counter, increasing by 1 for every allocated instance. Until you reach "NUM_MODULE_INSTANCES", where the constructor will return an out of memory error.
– Lundin
Aug 17 at 6:16
That's a fair point, @Lundin. That aspect of the design supposes that the indexes have inherent significance, which may or may not in fact be the case. It is the case, albeit trivially so, for the OP's starting case. Such significance, if it exists, could be further supported by providing a means to obtain an instance pointer without initializing.
– John Bollinger
Aug 17 at 11:51
So basically you reserve memory for n modules, no matter how many will be used, then return a pointer to the next unused element if the application initializes it. I guess that could work.
– L. Heinrichs
Aug 17 at 17:05
@L.Heinrichs Yes, since embedded systems are of a deterministic nature. There's no such thing as "endless amount of objects" nor "unknown amount of objects". Objects are often singletons too (hardware drivers), so there's often no need for the memory pool, as just one single instance of the object will exist.
– Lundin
Aug 20 at 14:06
I agree for most cases. The question had some theoretical interest as well. But I could use hundreds of 1-wire temperature sensors if there are enough IOs available (as the one example I can come up with now).
– L. Heinrichs
Aug 20 at 15:08
add a comment |Â
I think it's an odd design to pass the index from outside the class. When I implement memory pools like this, I let the index be a private counter, increasing by 1 for every allocated instance. Until you reach "NUM_MODULE_INSTANCES", where the constructor will return an out of memory error.
– Lundin
Aug 17 at 6:16
That's a fair point, @Lundin. That aspect of the design supposes that the indexes have inherent significance, which may or may not in fact be the case. It is the case, albeit trivially so, for the OP's starting case. Such significance, if it exists, could be further supported by providing a means to obtain an instance pointer without initializing.
– John Bollinger
Aug 17 at 11:51
So basically you reserve memory for n modules, no matter how many will be used, then return a pointer to the next unused element if the application initializes it. I guess that could work.
– L. Heinrichs
Aug 17 at 17:05
@L.Heinrichs Yes, since embedded systems are of a deterministic nature. There's no such thing as "endless amount of objects" nor "unknown amount of objects". Objects are often singletons too (hardware drivers), so there's often no need for the memory pool, as just one single instance of the object will exist.
– Lundin
Aug 20 at 14:06
I agree for most cases. The question had some theoretical interest as well. But I could use hundreds of 1-wire temperature sensors if there are enough IOs available (as the one example I can come up with now).
– L. Heinrichs
Aug 20 at 15:08
I think it's an odd design to pass the index from outside the class. When I implement memory pools like this, I let the index be a private counter, increasing by 1 for every allocated instance. Until you reach "NUM_MODULE_INSTANCES", where the constructor will return an out of memory error.
– Lundin
Aug 17 at 6:16
I think it's an odd design to pass the index from outside the class. When I implement memory pools like this, I let the index be a private counter, increasing by 1 for every allocated instance. Until you reach "NUM_MODULE_INSTANCES", where the constructor will return an out of memory error.
– Lundin
Aug 17 at 6:16
That's a fair point, @Lundin. That aspect of the design supposes that the indexes have inherent significance, which may or may not in fact be the case. It is the case, albeit trivially so, for the OP's starting case. Such significance, if it exists, could be further supported by providing a means to obtain an instance pointer without initializing.
– John Bollinger
Aug 17 at 11:51
That's a fair point, @Lundin. That aspect of the design supposes that the indexes have inherent significance, which may or may not in fact be the case. It is the case, albeit trivially so, for the OP's starting case. Such significance, if it exists, could be further supported by providing a means to obtain an instance pointer without initializing.
– John Bollinger
Aug 17 at 11:51
So basically you reserve memory for n modules, no matter how many will be used, then return a pointer to the next unused element if the application initializes it. I guess that could work.
– L. Heinrichs
Aug 17 at 17:05
So basically you reserve memory for n modules, no matter how many will be used, then return a pointer to the next unused element if the application initializes it. I guess that could work.
– L. Heinrichs
Aug 17 at 17:05
@L.Heinrichs Yes, since embedded systems are of a deterministic nature. There's no such thing as "endless amount of objects" nor "unknown amount of objects". Objects are often singletons too (hardware drivers), so there's often no need for the memory pool, as just one single instance of the object will exist.
– Lundin
Aug 20 at 14:06
@L.Heinrichs Yes, since embedded systems are of a deterministic nature. There's no such thing as "endless amount of objects" nor "unknown amount of objects". Objects are often singletons too (hardware drivers), so there's often no need for the memory pool, as just one single instance of the object will exist.
– Lundin
Aug 20 at 14:06
I agree for most cases. The question had some theoretical interest as well. But I could use hundreds of 1-wire temperature sensors if there are enough IOs available (as the one example I can come up with now).
– L. Heinrichs
Aug 20 at 15:08
I agree for most cases. The question had some theoretical interest as well. But I could use hundreds of 1-wire temperature sensors if there are enough IOs available (as the one example I can come up with now).
– L. Heinrichs
Aug 20 at 15:08
add a comment |Â
up vote
22
down vote
I program small micro-controllers in C++, which achieves exactly what you want.
What you call a module is a C++ class, it can contain data (either externally accessible or not) and functions (likewise). The constructor (a dedicated function) initilializes it. The constructor can take run-time parameters or (my favourite) compile-time (template) parameters. The functions within the class implicitly get the class variable as first parameter. (Or, often my preference, the class can act as a hidden singleton, so all data is accessed without this overhead).
The class object can be global (so you know at link-time that everything will fit), or stack-local, presumably in the main. (I don't like C++ globals because of the undefined global initialization order, so I prefer stack-local).
My preferred programming style is that modules are static classes, and their (static) configuration is by template parameters. This avoids nearly all overhad and enables optimization. Combine this with a tool that calculates the stack size and you can sleep without worries :)
My talk on this way of coding in C++: Objects? No Thanks!
A lot of embedded / microcontroller programmers seem to dislike C++ because they think it would force them to use all of C++. That is absolutely not neccessary, and would be a very bad idea. (You probably don't use all of C either! Think heap, floating point, setjmp/longjmp, printf, ...)
In a comment Adam Haun mentions RAII and initialization. IMO RAII has more to do with deconstruction, but his point is valid: global objects will be constructed before your main starts, so they might work on invalid assumptions (like a main clock speed that will be changed lateron). That is one more reason NOT to use global code-initialized objects. (I use a linker script that will fail when I have global code-initialized objects.) IMO such 'objects' should be explicitly created and passed around. This includes a 'waiting' facility 'object' that provides a wait() function. In my setup this is 'object' that sets the clock speed of the chip.
Talking about RAII: that is one more C++ feature that is very usefull in small embedded systems, although not for the reason (memory deallocation) it most used for in largere systems (small embedded systems mostly don't use dynamic memory deallocation). Think of locking a resource: you can make the locked resource a wrapper object, and restrict access to the resource to be only possible via the locking wrapper. When the wrapper goes out of scope, the resource is unlocked. This prevents access without locking, and makes it much more unlikely to forget the unlocking. with some (template) magic it can be zero-overhead.
The original question didn't mention C, hence my C++-centric answer. If it realy must be C....
You could use macro trickery: declare your stucts publicly, so they have a type and can be allocated globally, but mangle the names of their components beyond usability, unless some macro is defined differently, which is the case in your module's .c file. For extra security you could use the compile time in the mangling.
Or have a public version of your struct that has nothing usefull in it, and have the private version (with usefull data) only in your .c file, and assert that they are the same size. A bit of make-file trickery could automate this.
@Lundins comment about bad (embedded) programmers:
The type of programmer you describe would probably make a mess in any language. Macro's (present in C and C++) are one obvious way.
Tooling can help to some extent. For my students I mandate a built script that specifies no-exceptions, no-rtti, and gives a linker error when either the heap is used or code-initialized globals are present. And it specifies warning=error and enables nearly all warnings.
I encourage using templates, but with constexpr and concepts metaprogramming is less and less required.
"confused Arduino programmers" I would very much like to replace the Arduino (wiring, replication of code in libraries) programming style with a modern C++ approach, which can be easiere, more secure, and produce faster and smaller code. If only I had the time and power....
Thanks for this answer! Using C++ is an option, but we are using C in my company (what I havent mentioned explicitly). I have updated the question to let people know Im talking about C.
– L. Heinrichs
Aug 15 at 14:51
Why are you using (only) C? Maybe this gives you chance to convince them to at least consider C++... What you want is essentialy (a small part of) C++ realised in C.
– Wouter van Ooijen
Aug 15 at 14:55
What I do in my (first 'real' embedded hobby project) is initializing simple default in the constructor, and use a separate Init method for relevant classes. Another benefit is that I can pass stub pointers for unit testing.
– Michel Keijzers
Aug 15 at 15:00
2
@Michel for a hobby project you are free to choose the language? Take C++!
– Wouter van Ooijen
Aug 15 at 15:02
4
And while it is indeed perfectly possible to write good C++ programs for embedded, the problem is that some >50% of all embedded systems programmers out there are quacks, confused PC programmers, Arduino hobbyists etc etc. These kind of people simply cannot use a clean subset of C++, even if you explain it to their face. Give them C++ and before you know it, they will use the whole of STL, template metaprogramming, exception handling, multiple inheritance and so on. And the result is of course complete trash. This is, sadly, how some 8 out of 10 embedded C++ projects end up.
– Lundin
Aug 16 at 15:09
 |Â
show 9 more comments
up vote
22
down vote
I program small micro-controllers in C++, which achieves exactly what you want.
What you call a module is a C++ class, it can contain data (either externally accessible or not) and functions (likewise). The constructor (a dedicated function) initilializes it. The constructor can take run-time parameters or (my favourite) compile-time (template) parameters. The functions within the class implicitly get the class variable as first parameter. (Or, often my preference, the class can act as a hidden singleton, so all data is accessed without this overhead).
The class object can be global (so you know at link-time that everything will fit), or stack-local, presumably in the main. (I don't like C++ globals because of the undefined global initialization order, so I prefer stack-local).
My preferred programming style is that modules are static classes, and their (static) configuration is by template parameters. This avoids nearly all overhad and enables optimization. Combine this with a tool that calculates the stack size and you can sleep without worries :)
My talk on this way of coding in C++: Objects? No Thanks!
A lot of embedded / microcontroller programmers seem to dislike C++ because they think it would force them to use all of C++. That is absolutely not neccessary, and would be a very bad idea. (You probably don't use all of C either! Think heap, floating point, setjmp/longjmp, printf, ...)
In a comment Adam Haun mentions RAII and initialization. IMO RAII has more to do with deconstruction, but his point is valid: global objects will be constructed before your main starts, so they might work on invalid assumptions (like a main clock speed that will be changed lateron). That is one more reason NOT to use global code-initialized objects. (I use a linker script that will fail when I have global code-initialized objects.) IMO such 'objects' should be explicitly created and passed around. This includes a 'waiting' facility 'object' that provides a wait() function. In my setup this is 'object' that sets the clock speed of the chip.
Talking about RAII: that is one more C++ feature that is very usefull in small embedded systems, although not for the reason (memory deallocation) it most used for in largere systems (small embedded systems mostly don't use dynamic memory deallocation). Think of locking a resource: you can make the locked resource a wrapper object, and restrict access to the resource to be only possible via the locking wrapper. When the wrapper goes out of scope, the resource is unlocked. This prevents access without locking, and makes it much more unlikely to forget the unlocking. with some (template) magic it can be zero-overhead.
The original question didn't mention C, hence my C++-centric answer. If it realy must be C....
You could use macro trickery: declare your stucts publicly, so they have a type and can be allocated globally, but mangle the names of their components beyond usability, unless some macro is defined differently, which is the case in your module's .c file. For extra security you could use the compile time in the mangling.
Or have a public version of your struct that has nothing usefull in it, and have the private version (with usefull data) only in your .c file, and assert that they are the same size. A bit of make-file trickery could automate this.
@Lundins comment about bad (embedded) programmers:
The type of programmer you describe would probably make a mess in any language. Macro's (present in C and C++) are one obvious way.
Tooling can help to some extent. For my students I mandate a built script that specifies no-exceptions, no-rtti, and gives a linker error when either the heap is used or code-initialized globals are present. And it specifies warning=error and enables nearly all warnings.
I encourage using templates, but with constexpr and concepts metaprogramming is less and less required.
"confused Arduino programmers" I would very much like to replace the Arduino (wiring, replication of code in libraries) programming style with a modern C++ approach, which can be easiere, more secure, and produce faster and smaller code. If only I had the time and power....
Thanks for this answer! Using C++ is an option, but we are using C in my company (what I havent mentioned explicitly). I have updated the question to let people know Im talking about C.
– L. Heinrichs
Aug 15 at 14:51
Why are you using (only) C? Maybe this gives you chance to convince them to at least consider C++... What you want is essentialy (a small part of) C++ realised in C.
– Wouter van Ooijen
Aug 15 at 14:55
What I do in my (first 'real' embedded hobby project) is initializing simple default in the constructor, and use a separate Init method for relevant classes. Another benefit is that I can pass stub pointers for unit testing.
– Michel Keijzers
Aug 15 at 15:00
2
@Michel for a hobby project you are free to choose the language? Take C++!
– Wouter van Ooijen
Aug 15 at 15:02
4
And while it is indeed perfectly possible to write good C++ programs for embedded, the problem is that some >50% of all embedded systems programmers out there are quacks, confused PC programmers, Arduino hobbyists etc etc. These kind of people simply cannot use a clean subset of C++, even if you explain it to their face. Give them C++ and before you know it, they will use the whole of STL, template metaprogramming, exception handling, multiple inheritance and so on. And the result is of course complete trash. This is, sadly, how some 8 out of 10 embedded C++ projects end up.
– Lundin
Aug 16 at 15:09
 |Â
show 9 more comments
up vote
22
down vote
up vote
22
down vote
I program small micro-controllers in C++, which achieves exactly what you want.
What you call a module is a C++ class, it can contain data (either externally accessible or not) and functions (likewise). The constructor (a dedicated function) initilializes it. The constructor can take run-time parameters or (my favourite) compile-time (template) parameters. The functions within the class implicitly get the class variable as first parameter. (Or, often my preference, the class can act as a hidden singleton, so all data is accessed without this overhead).
The class object can be global (so you know at link-time that everything will fit), or stack-local, presumably in the main. (I don't like C++ globals because of the undefined global initialization order, so I prefer stack-local).
My preferred programming style is that modules are static classes, and their (static) configuration is by template parameters. This avoids nearly all overhad and enables optimization. Combine this with a tool that calculates the stack size and you can sleep without worries :)
My talk on this way of coding in C++: Objects? No Thanks!
A lot of embedded / microcontroller programmers seem to dislike C++ because they think it would force them to use all of C++. That is absolutely not neccessary, and would be a very bad idea. (You probably don't use all of C either! Think heap, floating point, setjmp/longjmp, printf, ...)
In a comment Adam Haun mentions RAII and initialization. IMO RAII has more to do with deconstruction, but his point is valid: global objects will be constructed before your main starts, so they might work on invalid assumptions (like a main clock speed that will be changed lateron). That is one more reason NOT to use global code-initialized objects. (I use a linker script that will fail when I have global code-initialized objects.) IMO such 'objects' should be explicitly created and passed around. This includes a 'waiting' facility 'object' that provides a wait() function. In my setup this is 'object' that sets the clock speed of the chip.
Talking about RAII: that is one more C++ feature that is very usefull in small embedded systems, although not for the reason (memory deallocation) it most used for in largere systems (small embedded systems mostly don't use dynamic memory deallocation). Think of locking a resource: you can make the locked resource a wrapper object, and restrict access to the resource to be only possible via the locking wrapper. When the wrapper goes out of scope, the resource is unlocked. This prevents access without locking, and makes it much more unlikely to forget the unlocking. with some (template) magic it can be zero-overhead.
The original question didn't mention C, hence my C++-centric answer. If it realy must be C....
You could use macro trickery: declare your stucts publicly, so they have a type and can be allocated globally, but mangle the names of their components beyond usability, unless some macro is defined differently, which is the case in your module's .c file. For extra security you could use the compile time in the mangling.
Or have a public version of your struct that has nothing usefull in it, and have the private version (with usefull data) only in your .c file, and assert that they are the same size. A bit of make-file trickery could automate this.
@Lundins comment about bad (embedded) programmers:
The type of programmer you describe would probably make a mess in any language. Macro's (present in C and C++) are one obvious way.
Tooling can help to some extent. For my students I mandate a built script that specifies no-exceptions, no-rtti, and gives a linker error when either the heap is used or code-initialized globals are present. And it specifies warning=error and enables nearly all warnings.
I encourage using templates, but with constexpr and concepts metaprogramming is less and less required.
"confused Arduino programmers" I would very much like to replace the Arduino (wiring, replication of code in libraries) programming style with a modern C++ approach, which can be easiere, more secure, and produce faster and smaller code. If only I had the time and power....
I program small micro-controllers in C++, which achieves exactly what you want.
What you call a module is a C++ class, it can contain data (either externally accessible or not) and functions (likewise). The constructor (a dedicated function) initilializes it. The constructor can take run-time parameters or (my favourite) compile-time (template) parameters. The functions within the class implicitly get the class variable as first parameter. (Or, often my preference, the class can act as a hidden singleton, so all data is accessed without this overhead).
The class object can be global (so you know at link-time that everything will fit), or stack-local, presumably in the main. (I don't like C++ globals because of the undefined global initialization order, so I prefer stack-local).
My preferred programming style is that modules are static classes, and their (static) configuration is by template parameters. This avoids nearly all overhad and enables optimization. Combine this with a tool that calculates the stack size and you can sleep without worries :)
My talk on this way of coding in C++: Objects? No Thanks!
A lot of embedded / microcontroller programmers seem to dislike C++ because they think it would force them to use all of C++. That is absolutely not neccessary, and would be a very bad idea. (You probably don't use all of C either! Think heap, floating point, setjmp/longjmp, printf, ...)
In a comment Adam Haun mentions RAII and initialization. IMO RAII has more to do with deconstruction, but his point is valid: global objects will be constructed before your main starts, so they might work on invalid assumptions (like a main clock speed that will be changed lateron). That is one more reason NOT to use global code-initialized objects. (I use a linker script that will fail when I have global code-initialized objects.) IMO such 'objects' should be explicitly created and passed around. This includes a 'waiting' facility 'object' that provides a wait() function. In my setup this is 'object' that sets the clock speed of the chip.
Talking about RAII: that is one more C++ feature that is very usefull in small embedded systems, although not for the reason (memory deallocation) it most used for in largere systems (small embedded systems mostly don't use dynamic memory deallocation). Think of locking a resource: you can make the locked resource a wrapper object, and restrict access to the resource to be only possible via the locking wrapper. When the wrapper goes out of scope, the resource is unlocked. This prevents access without locking, and makes it much more unlikely to forget the unlocking. with some (template) magic it can be zero-overhead.
The original question didn't mention C, hence my C++-centric answer. If it realy must be C....
You could use macro trickery: declare your stucts publicly, so they have a type and can be allocated globally, but mangle the names of their components beyond usability, unless some macro is defined differently, which is the case in your module's .c file. For extra security you could use the compile time in the mangling.
Or have a public version of your struct that has nothing usefull in it, and have the private version (with usefull data) only in your .c file, and assert that they are the same size. A bit of make-file trickery could automate this.
@Lundins comment about bad (embedded) programmers:
The type of programmer you describe would probably make a mess in any language. Macro's (present in C and C++) are one obvious way.
Tooling can help to some extent. For my students I mandate a built script that specifies no-exceptions, no-rtti, and gives a linker error when either the heap is used or code-initialized globals are present. And it specifies warning=error and enables nearly all warnings.
I encourage using templates, but with constexpr and concepts metaprogramming is less and less required.
"confused Arduino programmers" I would very much like to replace the Arduino (wiring, replication of code in libraries) programming style with a modern C++ approach, which can be easiere, more secure, and produce faster and smaller code. If only I had the time and power....
edited Aug 28 at 15:19
answered Aug 15 at 14:46
Wouter van Ooijen
43.3k150114
43.3k150114
Thanks for this answer! Using C++ is an option, but we are using C in my company (what I havent mentioned explicitly). I have updated the question to let people know Im talking about C.
– L. Heinrichs
Aug 15 at 14:51
Why are you using (only) C? Maybe this gives you chance to convince them to at least consider C++... What you want is essentialy (a small part of) C++ realised in C.
– Wouter van Ooijen
Aug 15 at 14:55
What I do in my (first 'real' embedded hobby project) is initializing simple default in the constructor, and use a separate Init method for relevant classes. Another benefit is that I can pass stub pointers for unit testing.
– Michel Keijzers
Aug 15 at 15:00
2
@Michel for a hobby project you are free to choose the language? Take C++!
– Wouter van Ooijen
Aug 15 at 15:02
4
And while it is indeed perfectly possible to write good C++ programs for embedded, the problem is that some >50% of all embedded systems programmers out there are quacks, confused PC programmers, Arduino hobbyists etc etc. These kind of people simply cannot use a clean subset of C++, even if you explain it to their face. Give them C++ and before you know it, they will use the whole of STL, template metaprogramming, exception handling, multiple inheritance and so on. And the result is of course complete trash. This is, sadly, how some 8 out of 10 embedded C++ projects end up.
– Lundin
Aug 16 at 15:09
 |Â
show 9 more comments
Thanks for this answer! Using C++ is an option, but we are using C in my company (what I havent mentioned explicitly). I have updated the question to let people know Im talking about C.
– L. Heinrichs
Aug 15 at 14:51
Why are you using (only) C? Maybe this gives you chance to convince them to at least consider C++... What you want is essentialy (a small part of) C++ realised in C.
– Wouter van Ooijen
Aug 15 at 14:55
What I do in my (first 'real' embedded hobby project) is initializing simple default in the constructor, and use a separate Init method for relevant classes. Another benefit is that I can pass stub pointers for unit testing.
– Michel Keijzers
Aug 15 at 15:00
2
@Michel for a hobby project you are free to choose the language? Take C++!
– Wouter van Ooijen
Aug 15 at 15:02
4
And while it is indeed perfectly possible to write good C++ programs for embedded, the problem is that some >50% of all embedded systems programmers out there are quacks, confused PC programmers, Arduino hobbyists etc etc. These kind of people simply cannot use a clean subset of C++, even if you explain it to their face. Give them C++ and before you know it, they will use the whole of STL, template metaprogramming, exception handling, multiple inheritance and so on. And the result is of course complete trash. This is, sadly, how some 8 out of 10 embedded C++ projects end up.
– Lundin
Aug 16 at 15:09
Thanks for this answer! Using C++ is an option, but we are using C in my company (what I havent mentioned explicitly). I have updated the question to let people know Im talking about C.
– L. Heinrichs
Aug 15 at 14:51
Thanks for this answer! Using C++ is an option, but we are using C in my company (what I havent mentioned explicitly). I have updated the question to let people know Im talking about C.
– L. Heinrichs
Aug 15 at 14:51
Why are you using (only) C? Maybe this gives you chance to convince them to at least consider C++... What you want is essentialy (a small part of) C++ realised in C.
– Wouter van Ooijen
Aug 15 at 14:55
Why are you using (only) C? Maybe this gives you chance to convince them to at least consider C++... What you want is essentialy (a small part of) C++ realised in C.
– Wouter van Ooijen
Aug 15 at 14:55
What I do in my (first 'real' embedded hobby project) is initializing simple default in the constructor, and use a separate Init method for relevant classes. Another benefit is that I can pass stub pointers for unit testing.
– Michel Keijzers
Aug 15 at 15:00
What I do in my (first 'real' embedded hobby project) is initializing simple default in the constructor, and use a separate Init method for relevant classes. Another benefit is that I can pass stub pointers for unit testing.
– Michel Keijzers
Aug 15 at 15:00
2
2
@Michel for a hobby project you are free to choose the language? Take C++!
– Wouter van Ooijen
Aug 15 at 15:02
@Michel for a hobby project you are free to choose the language? Take C++!
– Wouter van Ooijen
Aug 15 at 15:02
4
4
And while it is indeed perfectly possible to write good C++ programs for embedded, the problem is that some >50% of all embedded systems programmers out there are quacks, confused PC programmers, Arduino hobbyists etc etc. These kind of people simply cannot use a clean subset of C++, even if you explain it to their face. Give them C++ and before you know it, they will use the whole of STL, template metaprogramming, exception handling, multiple inheritance and so on. And the result is of course complete trash. This is, sadly, how some 8 out of 10 embedded C++ projects end up.
– Lundin
Aug 16 at 15:09
And while it is indeed perfectly possible to write good C++ programs for embedded, the problem is that some >50% of all embedded systems programmers out there are quacks, confused PC programmers, Arduino hobbyists etc etc. These kind of people simply cannot use a clean subset of C++, even if you explain it to their face. Give them C++ and before you know it, they will use the whole of STL, template metaprogramming, exception handling, multiple inheritance and so on. And the result is of course complete trash. This is, sadly, how some 8 out of 10 embedded C++ projects end up.
– Lundin
Aug 16 at 15:09
 |Â
show 9 more comments
up vote
7
down vote
I believe FreeRTOS (maybe another OS?) does something like what you're looking for by defining 2 different versions of the struct.
The 'real' one, used internally by the OS functions, and a 'fake' one which is the same size as the 'real' one, but doesn't have any useful members inside (just a bunch of int dummy1
and similar).
Only the 'fake' struct is exposed outside of the OS code, and this is used to allocate memory to static instances of the struct.
Internally, when functions in the OS are called, they are passed the address of the external 'fake' struct as a handle, and this is then typecast as a pointer to a 'real' struct so the OS functions can do what they need to do.
Good idea, I guess i could use --- #define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)])) --- BUILD_BUG_ON(sizeof(real_struct) != sizeof(fake_struct)) ----
– L. Heinrichs
Aug 15 at 14:47
add a comment |Â
up vote
7
down vote
I believe FreeRTOS (maybe another OS?) does something like what you're looking for by defining 2 different versions of the struct.
The 'real' one, used internally by the OS functions, and a 'fake' one which is the same size as the 'real' one, but doesn't have any useful members inside (just a bunch of int dummy1
and similar).
Only the 'fake' struct is exposed outside of the OS code, and this is used to allocate memory to static instances of the struct.
Internally, when functions in the OS are called, they are passed the address of the external 'fake' struct as a handle, and this is then typecast as a pointer to a 'real' struct so the OS functions can do what they need to do.
Good idea, I guess i could use --- #define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)])) --- BUILD_BUG_ON(sizeof(real_struct) != sizeof(fake_struct)) ----
– L. Heinrichs
Aug 15 at 14:47
add a comment |Â
up vote
7
down vote
up vote
7
down vote
I believe FreeRTOS (maybe another OS?) does something like what you're looking for by defining 2 different versions of the struct.
The 'real' one, used internally by the OS functions, and a 'fake' one which is the same size as the 'real' one, but doesn't have any useful members inside (just a bunch of int dummy1
and similar).
Only the 'fake' struct is exposed outside of the OS code, and this is used to allocate memory to static instances of the struct.
Internally, when functions in the OS are called, they are passed the address of the external 'fake' struct as a handle, and this is then typecast as a pointer to a 'real' struct so the OS functions can do what they need to do.
I believe FreeRTOS (maybe another OS?) does something like what you're looking for by defining 2 different versions of the struct.
The 'real' one, used internally by the OS functions, and a 'fake' one which is the same size as the 'real' one, but doesn't have any useful members inside (just a bunch of int dummy1
and similar).
Only the 'fake' struct is exposed outside of the OS code, and this is used to allocate memory to static instances of the struct.
Internally, when functions in the OS are called, they are passed the address of the external 'fake' struct as a handle, and this is then typecast as a pointer to a 'real' struct so the OS functions can do what they need to do.
edited Aug 15 at 14:42
answered Aug 15 at 14:31


brhans
8,35221726
8,35221726
Good idea, I guess i could use --- #define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)])) --- BUILD_BUG_ON(sizeof(real_struct) != sizeof(fake_struct)) ----
– L. Heinrichs
Aug 15 at 14:47
add a comment |Â
Good idea, I guess i could use --- #define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)])) --- BUILD_BUG_ON(sizeof(real_struct) != sizeof(fake_struct)) ----
– L. Heinrichs
Aug 15 at 14:47
Good idea, I guess i could use --- #define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)])) --- BUILD_BUG_ON(sizeof(real_struct) != sizeof(fake_struct)) ----
– L. Heinrichs
Aug 15 at 14:47
Good idea, I guess i could use --- #define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)])) --- BUILD_BUG_ON(sizeof(real_struct) != sizeof(fake_struct)) ----
– L. Heinrichs
Aug 15 at 14:47
add a comment |Â
up vote
2
down vote
Anonymous struct, so the struct may only be altered by the use of provided interface functions
In my opinion, this is pointless. You can put a comment there, but no point trying to hide it further.
C will never provide such high isolation, even if there is no declaration for the struct, it will be easy to accidentally overwrite it with e.g. mistaken memcpy() or buffer overflow.
Instead, just give the struct a name and trust other people to write good code also. It will also make debugging easier when the struct has a name you can use to refer to it.
add a comment |Â
up vote
2
down vote
Anonymous struct, so the struct may only be altered by the use of provided interface functions
In my opinion, this is pointless. You can put a comment there, but no point trying to hide it further.
C will never provide such high isolation, even if there is no declaration for the struct, it will be easy to accidentally overwrite it with e.g. mistaken memcpy() or buffer overflow.
Instead, just give the struct a name and trust other people to write good code also. It will also make debugging easier when the struct has a name you can use to refer to it.
add a comment |Â
up vote
2
down vote
up vote
2
down vote
Anonymous struct, so the struct may only be altered by the use of provided interface functions
In my opinion, this is pointless. You can put a comment there, but no point trying to hide it further.
C will never provide such high isolation, even if there is no declaration for the struct, it will be easy to accidentally overwrite it with e.g. mistaken memcpy() or buffer overflow.
Instead, just give the struct a name and trust other people to write good code also. It will also make debugging easier when the struct has a name you can use to refer to it.
Anonymous struct, so the struct may only be altered by the use of provided interface functions
In my opinion, this is pointless. You can put a comment there, but no point trying to hide it further.
C will never provide such high isolation, even if there is no declaration for the struct, it will be easy to accidentally overwrite it with e.g. mistaken memcpy() or buffer overflow.
Instead, just give the struct a name and trust other people to write good code also. It will also make debugging easier when the struct has a name you can use to refer to it.
answered Aug 16 at 10:15
jpa
1,265611
1,265611
add a comment |Â
add a comment |Â
up vote
2
down vote
Pure SW questions are better asked at https://stackoverflow.com.
The concept with exposing a struct of incomplete type to caller, like you describe, is often called "opaque type" or "opaque pointers" - anonymous struct means something else entirely.
The problem with this is that the caller won't be able to allocate instances of the object, only pointers to it. On a PC you would use malloc
inside the objects "constructor", but malloc is a no-go in embedded systems.
So what you do in embedded is to provide a memory pool. You have a limited amount of RAM, so restricting the number of objects that can be created is usually not a problem.
See Static allocation of opaque data types over at SO.
Ou thank you for clarifying the naming confusion on my end, ill adjust the OP. I was thinking of going to stack overflow, but decided i would like to target embedded programmers specifically.
– L. Heinrichs
Aug 17 at 17:04
add a comment |Â
up vote
2
down vote
Pure SW questions are better asked at https://stackoverflow.com.
The concept with exposing a struct of incomplete type to caller, like you describe, is often called "opaque type" or "opaque pointers" - anonymous struct means something else entirely.
The problem with this is that the caller won't be able to allocate instances of the object, only pointers to it. On a PC you would use malloc
inside the objects "constructor", but malloc is a no-go in embedded systems.
So what you do in embedded is to provide a memory pool. You have a limited amount of RAM, so restricting the number of objects that can be created is usually not a problem.
See Static allocation of opaque data types over at SO.
Ou thank you for clarifying the naming confusion on my end, ill adjust the OP. I was thinking of going to stack overflow, but decided i would like to target embedded programmers specifically.
– L. Heinrichs
Aug 17 at 17:04
add a comment |Â
up vote
2
down vote
up vote
2
down vote
Pure SW questions are better asked at https://stackoverflow.com.
The concept with exposing a struct of incomplete type to caller, like you describe, is often called "opaque type" or "opaque pointers" - anonymous struct means something else entirely.
The problem with this is that the caller won't be able to allocate instances of the object, only pointers to it. On a PC you would use malloc
inside the objects "constructor", but malloc is a no-go in embedded systems.
So what you do in embedded is to provide a memory pool. You have a limited amount of RAM, so restricting the number of objects that can be created is usually not a problem.
See Static allocation of opaque data types over at SO.
Pure SW questions are better asked at https://stackoverflow.com.
The concept with exposing a struct of incomplete type to caller, like you describe, is often called "opaque type" or "opaque pointers" - anonymous struct means something else entirely.
The problem with this is that the caller won't be able to allocate instances of the object, only pointers to it. On a PC you would use malloc
inside the objects "constructor", but malloc is a no-go in embedded systems.
So what you do in embedded is to provide a memory pool. You have a limited amount of RAM, so restricting the number of objects that can be created is usually not a problem.
See Static allocation of opaque data types over at SO.
answered Aug 16 at 15:02
Lundin
3,301929
3,301929
Ou thank you for clarifying the naming confusion on my end, ill adjust the OP. I was thinking of going to stack overflow, but decided i would like to target embedded programmers specifically.
– L. Heinrichs
Aug 17 at 17:04
add a comment |Â
Ou thank you for clarifying the naming confusion on my end, ill adjust the OP. I was thinking of going to stack overflow, but decided i would like to target embedded programmers specifically.
– L. Heinrichs
Aug 17 at 17:04
Ou thank you for clarifying the naming confusion on my end, ill adjust the OP. I was thinking of going to stack overflow, but decided i would like to target embedded programmers specifically.
– L. Heinrichs
Aug 17 at 17:04
Ou thank you for clarifying the naming confusion on my end, ill adjust the OP. I was thinking of going to stack overflow, but decided i would like to target embedded programmers specifically.
– L. Heinrichs
Aug 17 at 17:04
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2felectronics.stackexchange.com%2fquestions%2f391178%2fpossibilities-for-allocating-memory-for-modular-firmware-design-in-c%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
12
Most embedded FW designers are avoiding the dynamic allocation to keep the memory usage deterministic and simple. Especially if it is bare-metal and does not have underlying OS to manage the memory.
– Eugene Sh.
Aug 15 at 14:25
Exactly, thats why I want the linker to do the allocations.
– L. Heinrichs
Aug 15 at 14:28
4
I'm not quite sure I understand... How could you have the memory allocated by the linker if you have a dynamic number of instances? That seems quite orthogonal to me.
– jcaron
Aug 15 at 17:13
Why not let the linker allocate one big memory pool, and do your own allocations from that, which also gives you the benefit of a zero-overhead allocator. You can make the pool object static to the file with the allocation function so it's private. In some of my code, I do all the allocations in the various init routines, then I print out afterward how much was allocated, so in the final production compile I set the pool to that exact size.
– Lee Daniel Crocker
Aug 15 at 23:58
2
If it's a compile-time decision, you could simply define the number in your Makefile or equivalent, and you're all set. The number would not be in the source of the module, but would be application specific, and you just use an instance number as a parameter.
– jcaron
Aug 16 at 14:49