These changes don't actually fix any leaks. They just make sure that
Valgrind will find pointers to data structures that remain allocated
at process exit, and thus not falsely complain about leaks. In
particular, we are trying to avoid situations where there is no
pointer to the beginning of an allocated block (except possibly
within the block itself, which Valgrind won't count).
* Because dynahash.c never frees hashtable storage except by deleting
the whole hashtable context, it doesn't bother to track the individual
blocks of elements allocated by element_alloc(). This results in
"possibly lost" complaints from Valgrind except when the first element
of each block is actively in use. (Otherwise it'll be on a freelist,
but very likely only reachable via "interior pointers" within element
blocks, which doesn't satisfy Valgrind.)
To fix, if we're building with USE_VALGRIND, expend an extra pointer's
worth of space in each element block so that we can chain them all
together from the HTAB header. Skip this in shared hashtables though:
Valgrind doesn't track those, and we'd need additional locking to make
it safe to manipulate a shared chain.
While here, update a comment obsoleted by
9c911ec06.
* Put the dlist_node fields of catctup and catclist structs first.
This ensures that the dlist pointers point to the starts of these
palloc blocks, and thus that Valgrind won't consider them
"possibly lost".
* The postmaster's PMChild structs and the autovac launcher's
avl_dbase structs also have the dlist_node-is-not-first problem,
but putting it first still wouldn't silence the warning because we
bulk-allocate those structs in an array, so that Valgrind sees a
single allocation. Commonly the first array element will be pointed
to only from some later element, so that the reference would be an
interior pointer even if it pointed to the array start. (This is the
same issue as for dynahash elements.) Since these are pretty simple
data structures, I don't feel too bad about faking out Valgrind by
just keeping a static pointer to the array start.
(This is all quite hacky, and it's not hard to imagine usages where
we'd need some other idea in order to have reasonable leak tracking of
structures that are only accessible via dlist_node lists. But these
changes seem to be enough to silence this class of leakage complaints
for the moment.)
* Free a couple of data structures manually near the end of an
autovacuum worker's run when USE_VALGRIND, and ensure that the final
vac_update_datfrozenxid() call is done in a non-permanent context.
This doesn't have any real effect on the process's total memory
consumption, since we're going to exit as soon as that last
transaction is done. But it does pacify Valgrind.
* Valgrind complains about the postmaster's socket-files and
lock-files lists being leaked, which we can silence by just
not nulling out the static pointers to them.
* Valgrind seems not to consider the global "environ" variable as
a valid root pointer; so when we allocate a new environment array,
it claims that data is leaked. To fix that, keep our own
statically-allocated copy of the pointer, similarly to the previous
item.
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/285483.
1746756246@sss.pgh.pa.us
(void) unlink(sock_path);
}
/* Since we're about to exit, no need to reclaim storage */
- sock_paths = NIL;
}
static dlist_head DatabaseList = DLIST_STATIC_INIT(DatabaseList);
static MemoryContext DatabaseListCxt = NULL;
+/*
+ * Dummy pointer to persuade Valgrind that we've not leaked the array of
+ * avl_dbase structs. Make it global to ensure the compiler doesn't
+ * optimize it away.
+ */
+#ifdef USE_VALGRIND
+extern avl_dbase *avl_dbase_array;
+avl_dbase *avl_dbase_array;
+#endif
+
/* Pointer to my own WorkerInfo, valid on each worker */
static WorkerInfo MyWorkerInfo = NULL;
/* put all the hash elements into an array */
dbary = palloc(nelems * sizeof(avl_dbase));
+ /* keep Valgrind quiet */
+#ifdef USE_VALGRIND
+ avl_dbase_array = dbary;
+#endif
i = 0;
hash_seq_init(&seq, dbhash);
/*
* We leak table_toast_map here (among other things), but since we're
- * going away soon, it's not a problem.
+ * going away soon, it's not a problem normally. But when using Valgrind,
+ * release some stuff to reduce complaints about leaked storage.
*/
+#ifdef USE_VALGRIND
+ hash_destroy(table_toast_map);
+ FreeTupleDesc(pg_class_desc);
+ if (bstrategy)
+ pfree(bstrategy);
+#endif
+
+ /* Run the rest in xact context, mainly to avoid Valgrind leak warnings */
+ MemoryContextSwitchTo(TopTransactionContext);
/*
* Update pg_database.datfrozenxid, and truncate pg_xact if possible. We
*/
dlist_head ActiveChildList;
+/*
+ * Dummy pointer to persuade Valgrind that we've not leaked the array of
+ * PMChild structs. Make it global to ensure the compiler doesn't
+ * optimize it away.
+ */
+#ifdef USE_VALGRIND
+extern PMChild *pmchild_array;
+PMChild *pmchild_array;
+#endif
+
+
/*
* MaxLivePostmasterChildren
*
for (int i = 0; i < BACKEND_NUM_TYPES; i++)
num_pmchild_slots += pmchild_pools[i].size;
- /* Initialize them */
+ /* Allocate enough slots, and make sure Valgrind doesn't complain */
slots = palloc(num_pmchild_slots * sizeof(PMChild));
+#ifdef USE_VALGRIND
+ pmchild_array = slots;
+#endif
+
+ /* Initialize them */
slotno = 0;
for (int btype = 0; btype < BACKEND_NUM_TYPES; btype++)
{
* lookup key's hash value as a partition number --- this will work because
* of the way calc_bucket() maps hash values to bucket numbers.
*
- * For hash tables in shared memory, the memory allocator function should
- * match malloc's semantics of returning NULL on failure. For hash tables
- * in local memory, we typically use palloc() which will throw error on
- * failure. The code in this file has to cope with both cases.
+ * The memory allocator function should match malloc's semantics of returning
+ * NULL on failure. (This is essential for hash tables in shared memory.
+ * For hash tables in local memory, we used to use palloc() which will throw
+ * error on failure; but we no longer do, so it's untested whether this
+ * module will still cope with that behavior.)
*
* dynahash.c provides support for these types of lookup keys:
*
#include "access/xact.h"
#include "common/hashfn.h"
+#include "lib/ilist.h"
#include "port/pg_bitutils.h"
#include "storage/shmem.h"
#include "storage/spin.h"
Size keysize; /* hash key length in bytes */
long ssize; /* segment size --- must be power of 2 */
int sshift; /* segment shift = log2(ssize) */
+
+ /*
+ * In a USE_VALGRIND build, non-shared hashtables keep an slist chain of
+ * all the element blocks they have allocated. This pacifies Valgrind,
+ * which would otherwise often claim that the element blocks are "possibly
+ * lost" for lack of any non-interior pointers to their starts.
+ */
+#ifdef USE_VALGRIND
+ slist_head element_blocks;
+#endif
};
/*
{
HASHHDR *hctl = hashp->hctl;
Size elementSize;
+ Size requestSize;
+ char *allocedBlock;
HASHELEMENT *firstElement;
HASHELEMENT *tmpElement;
HASHELEMENT *prevElement;
/* Each element has a HASHELEMENT header plus user data. */
elementSize = MAXALIGN(sizeof(HASHELEMENT)) + MAXALIGN(hctl->entrysize);
+ requestSize = nelem * elementSize;
+
+ /* Add space for slist_node list link if we need one. */
+#ifdef USE_VALGRIND
+ if (!hashp->isshared)
+ requestSize += MAXALIGN(sizeof(slist_node));
+#endif
+
+ /* Allocate the memory. */
CurrentDynaHashCxt = hashp->hcxt;
- firstElement = (HASHELEMENT *) hashp->alloc(nelem * elementSize);
+ allocedBlock = hashp->alloc(requestSize);
- if (!firstElement)
+ if (!allocedBlock)
return false;
+ /*
+ * If USE_VALGRIND, each allocated block of elements of a non-shared
+ * hashtable is chained into a list, so that Valgrind won't think it's
+ * been leaked.
+ */
+#ifdef USE_VALGRIND
+ if (hashp->isshared)
+ firstElement = (HASHELEMENT *) allocedBlock;
+ else
+ {
+ slist_push_head(&hashp->element_blocks, (slist_node *) allocedBlock);
+ firstElement = (HASHELEMENT *) (allocedBlock + MAXALIGN(sizeof(slist_node)));
+ }
+#else
+ firstElement = (HASHELEMENT *) allocedBlock;
+#endif
+
/* prepare to link all the new entries into the freelist */
prevElement = NULL;
tmpElement = firstElement;
/* Should we complain if the unlink fails? */
}
/* Since we're about to exit, no need to reclaim storage */
- lock_files = NIL;
/*
* Lock file removal should always be the last externally visible action
static int save_argc;
static char **save_argv;
+/*
+ * Valgrind seems not to consider the global "environ" variable as a valid
+ * root pointer; so when we allocate a new environment array, it claims that
+ * data is leaked. To fix that, keep our own statically-allocated copy of the
+ * pointer. (Oddly, this doesn't seem to be a problem for "argv".)
+ */
+#if defined(PS_USE_CLOBBER_ARGV) && defined(USE_VALGRIND)
+extern char **ps_status_new_environ;
+char **ps_status_new_environ;
+#endif
+
/*
* Call this early in startup to save the original argc/argv values.
}
new_environ[i] = NULL;
environ = new_environ;
+
+ /* See notes about Valgrind above. */
+#ifdef USE_VALGRIND
+ ps_status_new_environ = new_environ;
+#endif
}
/*
typedef struct catctup
{
+ /*
+ * Each tuple in a cache is a member of a dlist that stores the elements
+ * of its hash bucket. We keep each dlist in LRU order to speed repeated
+ * lookups. Keep the dlist_node field first so that Valgrind understands
+ * the struct is reachable.
+ */
+ dlist_node cache_elem; /* list member of per-bucket list */
+
int ct_magic; /* for identifying CatCTup entries */
#define CT_MAGIC 0x57261502
*/
Datum keys[CATCACHE_MAXKEYS];
- /*
- * Each tuple in a cache is a member of a dlist that stores the elements
- * of its hash bucket. We keep each dlist in LRU order to speed repeated
- * lookups.
- */
- dlist_node cache_elem; /* list member of per-bucket list */
-
/*
* A tuple marked "dead" must not be returned by subsequent searches.
* However, it won't be physically deleted from the cache until its
*/
typedef struct catclist
{
+ /*
+ * Keep the dlist_node field first so that Valgrind understands the struct
+ * is reachable.
+ */
+ dlist_node cache_elem; /* list member of per-catcache list */
+
int cl_magic; /* for identifying CatCList entries */
#define CL_MAGIC 0x52765103
uint32 hash_value; /* hash value for lookup keys */
- dlist_node cache_elem; /* list member of per-catcache list */
-
/*
* Lookup keys for the entry, with the first nkeys elements being valid.
* All by-reference are separately allocated.
projects / postgresql.git / commitdiff