Skip to content

Inception score is non deterministic #6

Open
Open
Inception score is non deterministic#6

Description

@dougsouza
dougsouza
opened on Mar 11, 2019

I ran IS for the birds dataset, I noticed it is non deterministic. For the same input, output scores change for different runs. I am not sure if the inception network is in test mode, maybe dropout is being applied?

Additionally, for the birds dataset, I got an IS of 5.1324 for the real data (training set) and 22.7079 for the real data (test set). Which I think is weird, I thought it would be the other way around. Any idea why is that?

Thanks

Activity

  1. mkumar10 commented on Sep 12, 2019

    @mkumar10
    mkumar10
    on Sep 12, 2019

    It's not cause the inception network is trained on the validation data. As if you see for birds the dataset is divided into 150 classes and 50 classes for training and test.

  2. ah-ke commented on Nov 13, 2021

    @ah-ke
    ah-ke
    on Nov 13, 2021

    I ran IS for the birds dataset, I noticed it is non deterministic. For the same input, output scores change for different runs. I am not sure if the inception network is in test mode, maybe dropout is being applied?

    Additionally, for the birds dataset, I got an IS of 5.1324 for the real data (training set) and 22.7079 for the real data (test set). Which I think is weird, I thought it would be the other way around. Any idea why is that?

    Thanks

    inception_finetuned_models.zip

  3. priyankaupadhyay090 commented on Feb 11, 2022

    @priyankaupadhyay090
    priyankaupadhyay090
    on Feb 11, 2022

    I ran IS for the birds dataset, I noticed it is non deterministic. For the same input, output scores change for different runs. I am not sure if the inception network is in test mode, maybe dropout is being applied?

    Additionally, for the birds dataset, I got an IS of 5.1324 for the real data (training set) and 22.7079 for the real data (test set). Which I think is weird, I thought it would be the other way around. Any idea why is that?

    Thanks

    @dougsouza and @mkumar10
    Hi, It seems like you got the file running to get the IS score. I am trying to run IS score for bird dataset but pre-trained inception model link is not accessible anymore.

    you can see here:
    Inception Model:
    StackGAN inception model for birds and flowers evaluation. Download and save it in the current folder -----> this link requires permission to access.

    Can you please share the trained model files with me? It would be a great help

    Thank you

  4. Ashores commented on Apr 3, 2022

    @Ashores
    Ashores
    on Apr 3, 2022

    @priyankaupadhyay090 @fuyou123
    Hi, I am also trying to run IS score for bird datasets but pre-trained inception model link is not accessible anymore. Do you get the pre-trained model, can you please share the trained model files with me? It would be a great help.
    Thank you very much

  5. priyankaupadhyay090 commented on Apr 3, 2022

    @priyankaupadhyay090
    priyankaupadhyay090
    on Apr 3, 2022

    @priyankaupadhyay090 @fuyou123 Hi, I am also trying to run IS score for bird datasets but pre-trained inception model link is not accessible anymore. Do you get the pre-trained model, can you please share the trained model files with me? It would be a great help. Thank you very much

    file is not avaialble here: https://drive.google.com/open?id=0B3y_msrWZaXLMzNMNWhWdW0zVWs&authuser=

  6. Ashores commented on Apr 4, 2022

    @Ashores
    Ashores
    on Apr 4, 2022

    I know the file is available in the URL, but this link requires permission to access, I can not get the permission. Do you get the file, can you please share the the file with me by email? It would be a great help.
    Thank you very much.
    @priyankaupadhyay090

  7. Ashores commented on Apr 5, 2022

    @Ashores
    Ashores
    on Apr 5, 2022

    My email is piperne96@gmail.com，Thank you very much @priyankaupadhyay090

  8. hannanyi commented on Apr 11, 2022

    @hannanyi
    hannanyi
    on Apr 11, 2022

    I know the file is available in the URL, but this link requires permission to access, I can not get the permission. Do you get the file, can you please share the the file with me by email? It would be a great help.
    Thank you very much.
    @priyankaupadhyay090

    that is weird because I can access it. I can upload the model on my drive and share the link. Send me your email address.

    hi, I can't download it either,Could you please send a copy to my email (hannan@stumail.nwu.edu.cn) in your spare time? Thank you very much.

  9. 2nite2 commented on Apr 14, 2022

    @2nite2
    2nite2
    on Apr 14, 2022

    My email is piperne96@gmail.com，Thank you very much @priyankaupadhyay090

    Hello, have you got the file？Could you please share it with me if it's convinient, my email is 2nite1116@gmail.com, thank you very much. @Ashores

  10. 2nite2 commented on Apr 14, 2022

    @2nite2
    2nite2
    on Apr 14, 2022

    I know the file is available in the URL, but this link requires permission to access, I can not get the permission. Do you get the file, can you please share the the file with me by email? It would be a great help.
    Thank you very much.
    @priyankaupadhyay090

    that is weird because I can access it. I can upload the model on my drive and share the link. Send me your email address.

    hi, I can't download it either,Could you please send a copy to my email (hannan@stumail.nwu.edu.cn) in your spare time? Thank you very much.

    Hello, have you got the file？Could you please share it with me if it's convinient, my email is 2nite1116@gmail.com, thank you very much. @hannanyi

  11. 2nite2 commented on Apr 14, 2022

    @2nite2
    2nite2
    on Apr 14, 2022

    @priyankaupadhyay090 @fuyou123 Hi, I am also trying to run IS score for bird datasets but pre-trained inception model link is not accessible anymore. Do you get the pre-trained model, can you please share the trained model files with me? It would be a great help. Thank you very much

    file is avaialble here: https://drive.google.com/open?id=0B3y_msrWZaXLMzNMNWhWdW0zVWs&authuser=0

    Hello, I have no permission either, Could you please share it with me if it's convinient? my email is 2nite1116@gmail.com, thank you very much! @priyankaupadhyay090

  12. 2nite2 commented on Apr 28, 2022

    @2nite2
    2nite2
    on Apr 28, 2022

    I know the file is available in the URL, but this link requires permission to access, I can not get the permission. Do you get the file, can you please share the the file with me by email? It would be a great help.
    Thank you very much.
    @priyankaupadhyay090

    that is weird because I can access it. I can upload the model on my drive and share the link. Send me your email address.

    hi, I can't download it either,Could you please send a copy to my email (hannan@stumail.nwu.edu.cn) in your spare time? Thank you very much.

    Hello, I received your email, but it is strange that there is no model attached, I'm sorry to trouble you again, could you please send it to me again? 2nite1116@gmail.com Thanks! @hannanyi

  13. hannanyi commented on Apr 28, 2022

    @hannanyi
    hannanyi
    on Apr 28, 2022 via email

  14. Ashores commented on Oct 11, 2022

    @Ashores
    Ashores
    on Oct 11, 2022 via email

  15. priyankaupadhyay090 commented on Oct 11, 2022

    @priyankaupadhyay090
    priyankaupadhyay090
    on Oct 11, 2022

    My email is @.*** Thank you very much Priyanka Upadhyay @.> 于2022年4月4日周一 15:50写道：
    …
    I know the file is available in the URL, but this link requires permission to access, I can not get the permission. Do you get the file, can you please share the the file with me by email? It would be a great help. Thank you very much. @priyankaupadhyay090 https://github.com/priyankaupadhyay090 that is weird because I can access it. I can upload the model on my drive and share the link. Send me your email address. — Reply to this email directly, view it on GitHub <#6 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AWJNAVMV4MYKMLLURP5R5JTVDKNMDANCNFSM4G5EBG7Q . You are receiving this because you commented.Message ID: @.    >

    Please ask the author, I don't have access to it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions