Dynamic Patchification in VLMs
Early vision-language architectures process visual inputs by forcing incoming images into fixed square grids. Standard Vision Transformers scale, crop, or stretch images to fit pre-defined canvas sizes. Forcing images into fixed dimensions harms model performance. Distorting aspect ratios warps spatial geometry, while downsampling destroys the fine text, small symbols, and table layouts critical for document understanding.
Instead of resizing the image, dynamic patchification splits it into a flexible grid of uniform tiles that preserve the original aspect ratio. This lets vision encoders process images at native resolutions without warping shapes or losing fine details.
Read More
| Share