Add support to prompt with a YouTube Video URL

[lucastanzi]

Description of the feature request:

In Vertex AI it is finally possible to attach a YouTube Video URL as an alternative to upload a video file. It would be nice to have feature parity here and be able to call something like:
model.generate_content(youtube-video-id='xxxx', "Summarize this video")

What problem are you trying to solve with this feature?

Be able to send a prompt using a YouTube Video URL/ID

Any other information you'd like to share?

No response

[sayakpaul]

Do we have an example with a local video already?

[MarkDaoust]

Yes, local video files work:

Using this SDK: https://github.com/google-gemini/cookbook/blob/28fc33fbc2189a30a682148165ea6049ffa93db0/quickstarts/Video.ipynb#L40

Using the new SDK: https://github.com/google-gemini/cookbook/blob/28fc33fbc2189a30a682148165ea6049ffa93db0/quickstarts/Video_understanding.ipynb

[sayakpaul]

How do I do it with this pattern?

from google import genai
from PIL import Image

client = genai.Client()

response = client.models.generate_content(
    model='gemini-2.0-flash',
    contents=[
        'Tell me a story based on this image',
        Image.open(image_path)
    ]
)
print(response.text)

The links you provided all need me to call upload_file() and then create a model from the client.

[MarkDaoust]

Oh, without a separate file upload.

try:

contents = [
  "Tell me a story aobut this video",
   types.Part.from_bytes(pathlib.Path('video.mp4').read_bytes(), mime_type="video/mp4")
]

Does that work?

[sayakpaul]

Full code:

from google import genai
import os
import pathlib
from google.genai import types


client = genai.Client(api_key=os.getenv("GEMINI_API_KEY"))

response = client.models.generate_content(
    model='gemini-2.0-flash',
    contents=[
        'Tell me a story based on this image',
        types.Part.from_bytes(data=pathlib.Path('assets%2Foutput_2.mp4').read_bytes(), mime_type="video/mp4")
    ]
)
print(response.text)

The asset used comes from https://huggingface.co/finetrainers/cakeify-v0.

Consider adding it to somewhere more discoverable.

Story

Okay, here's a story based on the images you sent:

**The Secret Garden Cake**

Agatha, a baker with a flair for the dramatic, stood before her masterpiece. It wasn't just any cake; it was "The Secret Garden Cake." Shaped like a vibrant red flowerpot, it sat proudly on a stark white plate, a testament to Agatha's hours of labor. The smooth, almost lacquered, exterior hid a world of delicious secrets within.

Tonight was the unveiling. Agatha was known for her inventive desserts, but this one was special. It was a gift for her granddaughter, Lily, who was recovering from a long illness and had found solace in tending to the small garden on their balcony.

Agatha donned her black gloves, a habit she'd picked up from watching too many cooking shows. She picked up the gleaming, serrated knife. A dramatic flourish, she thought.

The first slice was a revelation. The knife slid through the red fondant, revealing layers of rich chocolate cake, fluffy vanilla buttercream, and a hint of raspberry jam. A collective "ooh" escaped the small crowd of family gathered around the table.

"But Grandma," Lily whispered, her eyes wide with wonder, "it looks just like… my flowerpot!"

Agatha smiled, a twinkle in her eye. "That's because it is, my dear, but one with a very special ingredient - love."

As Lily took the first bite, a genuine smile bloomed on her face. The cake was more than just a treat; it was a reminder of the beauty and joy that could be found even in the smallest things, a secret garden of sweetness within a vibrant, red flowerpot. And Agatha knew, in that moment, that she had created something truly magical.

Interestingly, it seems like Gemini treats videos like images.

[demoncoder-crypto]

is this issue open, I would love to contribute

[MarkDaoust]

https://ai.google.dev/gemini-api/docs/vision?lang=python#youtube

Hi, the API just added support for this.

The docs all use the new SDK google.genai, but you can try following the same patterns with this SDK and it might work. let us know.