| View previous topic :: View next topic |
| Author |
Message |
chatgris
Guru
Joined: 14 Oct 2002
Posts: 383
Location: Canada
|
 Posted: Sun Aug 24, 2003 6:56 am Post subject: Overloading C++ new[] sizeof() (pretty advanced) |
 |
|
While overloading operator new[] in a class I realized that it was adding a 4 byte overhead to the allocation.. later I found that the size of the array was stored in that 4 byte space.. I have managed to retrieve it, and am wondering if there is a more clean way of extracting it.. code below.
| Code: | #include <iostream>
#include <fstream>
using namespace std;
class Test {
public:
void * operator new[] ( size_t size ) {
cout << "Allocating Array " << size << endl;
return malloc( size );
}
void operator delete[] ( void * ptr, size_t size ) {
cout << "Freeing array " << size << endl;
free( ptr );
}
private:
size_t tmp;
int value;
};
int main () {
Test * tmparray = new Test[20]( );
//The right answer, as I suspected it's stored in the first 4 bytes
//Must be an easier way to extract this value.
cout << *((size_t *)tmparray - 1) << endl;
//Sizeof of the pointer, 4 bytes on 32 bit
cout << sizeof( tmparray ) << endl;
//Sizeof the single class
cout << sizeof( *tmparray ) << endl;
delete[] tmparray;
return 0;
}
|
_________________
Open your mind. Open your source.
Due credit for avatar from http://www.aikida.net |
|
| Back to top |
|
 |
far
Guru
Joined: 10 Mar 2003
Posts: 394
Location: Stockholm, Sweden
|
| Posted: Sun Aug 24, 2003 9:47 am Post subject: |
|
|
If sizeof(Test) is 8 bytes then tmparray-1 is 8 bytes in advance of tmparray.
Anyway, I would think this is implementation dependent.
_________________
The Porthole Portage Frontend |
|
| Back to top |
|
|
zhenlin
Veteran
Joined: 09 Nov 2002
Posts: 1361
|
| Posted: Sun Aug 24, 2003 10:18 am Post subject: |
|
|
If it isn't it the standard [which is massive], it's an extension.
It isn't listed under the GCC extensions though.
To get the size, correctly, using this extension:
| Code: |
*((size_t)(array - 1) + 1)
|
So it seems. The first size_t stores something strange... |
|
| Back to top |
|
|
chatgris
Guru
Joined: 14 Oct 2002
Posts: 383
Location: Canada
|
| Posted: Sun Aug 24, 2003 12:27 pm Post subject: |
|
|
| far wrote: | If sizeof(Test) is 8 bytes then tmparray-1 is 8 bytes in advance of tmparray.
Anyway, I would think this is implementation dependent. |
I've casted it as a size_t * before I did the subtraction.
_________________
Open your mind. Open your source.
Due credit for avatar from http://www.aikida.net |
|
| Back to top |
|
|
quixoticsycophant
n00b
Joined: 10 Jan 2003
Posts: 16
|
| Posted: Sun Aug 24, 2003 2:44 pm Post subject: Re: Overloading C++ new[] sizeof() (pretty advanced) |
|
|
| chatgris wrote: | While overloading operator new[] in a class I realized that it was adding a 4 byte overhead to the allocation.. later I found that the size of the array was stored in that 4 byte space.. I have managed to retrieve it, and am wondering if there is a more clean way of extracting it.. code below.
|
 
Would you mind if I asked what your motivation is?
I ask because, at every angle, I think you really don't want to do this.
(1) std::vector<> is fast, easy to use, and makes your code more general because it will use a standard API for containers. It is fast because the standard allocators are already super-optimized.
(2) If you really do come up with a better allocator scheme for some particular purpose, you can just let std::vector<> use it (the second optional template parameter is an allocator).
(3) As others have pointed out, the implementation of malloc is (suprise surprise) implementation-dependent. All std::vector<> implementations I've seen store an "end" pointer or a size integer, and for good reason.
Occasionally there is good reason to make something implementation- or architecture-dependant, for example if there is a trick which will give a significant performance boost and if such a boost is necessary. But what you have outlined here would be much slower than std::vector<>.
I don't mean to sound harsh -- I give you credit for "decoding" the gcc malloc scheme. But otherwise you appear to be driving toward a very steep and rocky cliff  |
|
| Back to top |
|
|
chatgris
Guru
Joined: 14 Oct 2002
Posts: 383
Location: Canada
|
| Posted: Sun Aug 24, 2003 3:49 pm Post subject: |
|
|
Haha. I can understand your puzzlement at my motives, and I would be questioning the same had I seen this post 
The thing is that I am writing a system whose objective is to conserve as much RAM as possible.
I have built a fixed size allocator//manager and I would like to be able to use new[struct].
it would be fine if new[struct] returned JUST what malloc would, because I would store the array amount myself.
However, instead it looks like I will probably use a direct call to my allocator instead of turning the struct into a class and neatly overriding the new operator.
I wouldn't actually do what I'm doing there because I know it's not standard.. however just as proof that the size was being stored I worked that out to avoid the "It's not possible to get the size of the array, it's pure memory" answers that I surely would have gotten without it
And yes, vectors are very useful and used within this program, just very not suitable for this particular situation where I'm allocating possible hundreds of thousands of small arrays.
Thanks for the concern though 
_________________
Open your mind. Open your source.
Due credit for avatar from http://www.aikida.net |
|
| Back to top |
|
|
chatgris
Guru
Joined: 14 Oct 2002
Posts: 383
Location: Canada
|
| Posted: Sun Aug 24, 2003 3:52 pm Post subject: |
|
|
| Quote: | | I give you credit for "decoding" the gcc malloc scheme |
Oh yeah, by the way this isn't the malloc scheme I believe... I overloaded operator new[], so it's just asking me for memory. Any memory I return via malloc would already have the 4-8 byte overhead this is something extra I believe.
Josh.
_________________
Open your mind. Open your source.
Due credit for avatar from http://www.aikida.net |
|
| Back to top |
|
|
quixoticsycophant
n00b
Joined: 10 Jan 2003
Posts: 16
|
| Posted: Sun Aug 24, 2003 10:19 pm Post subject: |
|
|
| chatgris wrote: | | And yes, vectors are very useful and used within this program, just very not suitable for this particular situation where I'm allocating possible hundreds of thousands of small arrays. |
Ah I see that makes sense. I didn't have a context to your post so obviously I assumed you didn't know what you were doing . In hindsight I could have noticed your "guru" status and looked at your posting history...
To clarify:o malloc() returns a chunk of memory with a header containing the size of the chunk.
o built-in operator new[]() returns a chunk of memory with a header containing the size of the chunk AND the size of the class. It needs to know the class size in order to call the destructors for each item in the array.
o for classes without a destructor, most (all?) compilers (like gcc) will make operator new[]() behave just like malloc.
So then what about an overloaded operator new[]()?
An overload of operator new[]() implies an overload of operator delete[](). Therefore, it seems to me, the programmer is wholly responsible for deleting the entire array. The whole point of overloading these operators is to take that responsibility away from the implementation.
I am very concerned about what you have found, though. I have verified that operator new[]() does indeed modify, on its own, 4 bytes BEFORE the pointer that you return from it. That can't be right.
Suppose you return a pointer to the beginning of your pool. If your pool is obtained from a malloc() then your malloc header will get corrupted and you'll have a serious problem on your hands. If you have a static pool then it's a seg fault.
And if you are required to compensate for those 4 bytes then you are required to know the implementation of operator new[]() before overloading operator new[](), and that can't be right either.
I'd have to research it more, but right now it looks like a gcc bug.
In any case, operator new[]() sucks anyway and nobody should ever use it. I've seen too many memory leaks due to someone calling "delete foo" when they should have called "delete [] foo".
-qs
Last edited by quixoticsycophant on Sun Aug 24, 2003 10:27 pm; edited 1 time in total |
|
| Back to top |
|
|
pygoscelis
Guru
Joined: 07 Jun 2003
Posts: 402
|
| Posted: Sun Aug 24, 2003 10:22 pm Post subject: |
|
|
| If you want to do it safely and portably, do NOT use yourclass::operator new[]. Use yourclass::allocate_array(). Such function would call whatever allocator it uses for raw memory, then call placement new in a loop for array elements. Do not forget to handle throwing constructors |
|
| Back to top |
|
|
chatgris
Guru
Joined: 14 Oct 2002
Posts: 383
Location: Canada
|
| Posted: Wed Aug 27, 2003 2:15 pm Post subject: |
|
|
Could you possibly post or point me to some example code for that? I have never heard of any standard hook called allocate_array.
Thanks, Josh.
_________________
Open your mind. Open your source.
Due credit for avatar from http://www.aikida.net |
|
| Back to top |
|
|
pygoscelis
Guru
Joined: 07 Jun 2003
Posts: 402
|
| Posted: Wed Aug 27, 2003 10:07 pm Post subject: |
|
|
| chatgris wrote: | Could you possibly post or point me to some example code for that? I have never heard of any standard hook called allocate_array.
Thanks, Josh. |
There's no such standard hook indeed. I tried to say that your custom needs cannot be served by overloading operator new[], so you have to use some other method of allocating arrays of your class type. It doesn't matter how do you call the function that will do it.
It does not need to be standard because you only use it to implement a class that wraps raw arrays of your type. Provided that boost::array doesn't suit your needs, that is |
|
| Back to top |
|
|
quixoticsycophant
n00b
Joined: 10 Jan 2003
Posts: 16
|
| Posted: Thu Aug 28, 2003 12:23 am Post subject: |
|
|
| Quote: | There's no such standard hook indeed. I tried to say that your custom needs cannot be served by overloading operator new[], so you have to use some other method of allocating arrays of your class type. It doesn't matter how do you call the function that will do it.
|
By the way, C++ does have standard functions for this type of thing. These are unitialized_copy(), unintialized_fill(), uninitialized_fill_n() and such. Allocators also have a generic ::allocate() function. In addition, there is get_temporary_buffer() and return_temporary_buffer().
The standard library functions contain potentially signifcant optimizations for a given platform, so it is well worth getting a handle on them.
-qs |
|
| Back to top |
|
|
|
Portage & Programming
