https://huggingface.co/docs/transformers.js/en/index
Functional equivalent of Hugging Face Python Transformers.
Modified from https://huggingface.co/docs/transformers.js/main/en/tutorials/vanilla-js
Analyzes an image and attempts to label it using Xenova/detr-resnet-50.
Code is here
https://github.com/dsteele101/imageDetect
Next Steps:
Fix bounding boxes
Integrate with GPT client
Explore other models to get color, shapes, etc from images
Use that model data to pass into GPT and get a reasonable description of the image
Example prompt:
Write a description of an image based on the following information returned from a machine learning model about an image.
Cat, grey, short fur, green eyes, laying down
Example Response: In this captivating image, a graceful cat with sleek, short grey fur commands attention as it lounges in a relaxed pose. The feline's striking green eyes add a touch of mystery and depth to its visage, creating a mesmerizing gaze that captures the viewer's curiosity. The cat's calm demeanor is beautifully conveyed through its leisurely position, creating a serene atmosphere. The subtle interplay of light and shadow accentuates the texture of its fur, highlighting the elegance and poise of this enchanting grey cat.
Explore integration options with Amelia.ai