Skip to content

Decrease Markov network memory footprint #413

Open
Open
Decrease Markov network memory footprint#413
Labels
kind:refactor

Description

@ForNeVeR
ForNeVeR
opened on Nov 23, 2016

I believe that currently Markov network images are consuming more memory that it's needed for them. We need to profile Horta memory and decrease network memory footprint if that's possible.

Activity

  1. ForNeVeR
    added
    kind:refactor
    on Nov 23, 2016

  2. ForNeVeR commented on Nov 27, 2016

    @ForNeVeR
    ForNeVeR
    on Nov 27, 2016
    MemberAuthor

    Please first of all check #134 when implementing.

  3. ForNeVeR commented on Dec 4, 2016

    @ForNeVeR
    ForNeVeR
    on Dec 4, 2016
    MemberAuthor

    Some optimization points (things to consider):

    1. Intern strings (words) after parsing the file.
    2. Intern phrase vectors inside Platonus library.
    3. Make a single parser actor (so it won't eat too much memory while parsing multiple networks at once).
    4. Progressively load strings from the database (don't load them all at once).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind:refactor

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions