Databases

NAVIGATION
CATEGORIES
REFERRENCE
LINKS
  • memory leak under heavy load?

    6 answers - 2013 bytes - related search similar search Add To My Delicious Add To My Stumble Upon Add To My Google Mark Add To My Facebook Add To My Digg Add To My Reddit

    hi
    i think i've encountered a bug in postgresql 8.1.
    yet - i'm not reallty info submitting it to -bugs, as i have no way to
    successfully redo it again.
    basically
    i have server, with dual opteron, 4g of memory, 2gb of swap.
    everything working under centos 4.2.
    what i say is that postmaster user started to "eat" memory.
    it allocated *all* memory (both ram and swap), and then died.
    load on the machine jumped to something around 20.
    I noticed a similar occurrence. We have a high-load PostgreSQL database
    -- not a ridiculous amount of inserts or updates, but a huge variety of
    diverse queries on some 200 tables.
    We had noticed load averages of 3-4 on our database for the past couple
    days. Then, this morning, Postgres got killed twice by the Linux
    out-of-memory process killer. (Also on a dual , 4GB of memory.)
    We were showing 3.5 GB of memory allocated to *something*, but stopping
    Postgres completely for a few seconds didn't lower the number. It wasn't
    taken by any process, which leads me to believe that it's a kernel bug.
    reboot later, everything is rosy -- load hovers around 1.2, there's
    enough free memory to have a 2.5 GB buffer cache, and swap is untouched.
    PostgreSQL 7.4 had run on this box flawlessly for six months -- bad RAM
    forced us to take it down -- then again for another month until we
    upgraded to 8.1 last week. Like the original poster, we're set up for
    ~500 MB of shared memory; certainly not enough to make the kernel kill
    -9 postmaster. Kernel is 2.6.11-gentoo-r6, same as before the upgrade.
    Also, this didn't happen in our test environment, which uses a similar
    but x86 server. Perhaps this is AMD64 related?
    Glynn
    Freedom Healthcare
    (end of broadcast)
    TIP 9: In versions below 8.0, the planner will ignore your desire to
    choose an index scan if your joining column's datatypes do not
    match
  • No.1 | | 919 bytes | |

    Will Glynn <wglynn (AT) freedomhealthcare (DOT) orgwrote:
    Postgres completely for a few seconds didn't lower the number. It wasn't
    taken by any process, which leads me to believe that it's a kernel bug.

    If it was a shared memory segment allocated a particular way (I
    *think* it's "shm_open", I'm not 100% sure), it's not erronious for the
    kernel to leave it behind after all processes are gone see
    .

    If postgres needs this much shared memory and wants it to go away on
    a crash, I think (again, I'm a neophyte at this still, I havent even fixed
    mod_bt fo rthis yet) that an mmap()ed file is the way to go but then
    don't you need enough harddrive space to support your shared memory?

    I don't know, this whole things confusing
    - Tyler

    (end of broadcast)
    TIP 3: Have you checked our extensive FAQ?

  • No.2 | | 760 bytes | |

    Fri, Dec 02, 2005 at 03:53:20PM -0800, Tyler MacDonald wrote:
    Will Glynn <wglynn (AT) freedomhealthcare (DOT) orgwrote:
    Postgres completely for a few seconds didn't lower the number. It wasn't
    taken by any process, which leads me to believe that it's a kernel bug.

    If it was a shared memory segment allocated a particular way (I
    *think* it's "shm_open", I'm not 100% sure), it's not erronious for the
    kernel to leave it behind after all processes are gone see
    .

    But this shouldn't be an issue here. If you set the IPC_RMID flag then
    the kernel should remove the segment when all users go away. This is
    standard IPC behaviour and is documentated in the manpage

    Have a nice day,
  • No.3 | | 296 bytes | |

    But this shouldn't be an issue here. If you set the IPC_RMID flag then
    the kernel should remove the segment when all users go away. This is
    standard IPC behaviour and is documentated in the manpage
    Would you please tell me where to find the manpage and how to set IPC_RMID flag?
  • No.4 | | 513 bytes | |

    Thu, Dec 08, 2005 at 12:29:11PM +0800, Kathy Lo wrote:
    But this shouldn't be an issue here. If you set the IPC_RMID flag then
    the kernel should remove the segment when all users go away. This is
    standard IPC behaviour and is documentated in the manpage

    Would you please tell me where to find the manpage and how to set IPC_RMID flag?

    See the shmctl() manpage:

    int shmctl(int shmid, int cmd, struct shmid_ds *buf);

    of the command ids is IPC_RMID

    Have a nice day,
  • No.5 | | 669 bytes | |

    12/8/05, Martijn van <kleptog (AT) svana (DOT) orgwrote:
    Thu, Dec 08, 2005 at 12:29:11PM +0800, Kathy Lo wrote:
    But this shouldn't be an issue here. If you set the IPC_RMID flag then
    the kernel should remove the segment when all users go away. This is
    standard IPC behaviour and is documentated in the manpage
    --
    Would you please tell me where to find the manpage and how to set IPC_RMID
    flag?

    See the shmctl() manpage:

    int shmctl(int shmid, int cmd, struct shmid_ds *buf);

    of the command ids is IPC_RMID

    Do I need to change the source code of postgresql if I want to set
    IPC_RMID flag to solve this problem?
  • No.6 | | 730 bytes | |

    Fri, Dec 09, 2005 at 12:15:44PM +0800, Kathy Lo wrote:
    See the shmctl() manpage:

    int shmctl(int shmid, int cmd, struct shmid_ds *buf);

    of the command ids is IPC_RMID

    Do I need to change the source code of postgresql if I want to set
    IPC_RMID flag to solve this problem?

    No because it's completely unrelated. The amount of shared memory
    doesn't vary while the system is running and it gets removed once you
    restart postgres. So it can't cause you to run out of memory (unless
    you allocated a truly huge amount of memory that way, but that's bad
    for other reasons). At worst it's gets stuffed into swap until you next
    start postgres.

    Have a nice day,

Re: memory leak under heavy load?


max 4000 letters.
Your nickname that display:
In order to stop the spam: 2 + 1 =
QUESTION ON "Databases"

EMSDN.COM