While it will vary a bit with latitude, bucks start growing new antlers along in May, mas o menos. The size is a function of diet and age. They shed along in January, usually.
Near Austin, Texas, I shot a big doe one year just before Christmas. Imagine my surprise when I went to field dress the critter, and found non-doelike appendages! The buck had already shed!
I have an April, 1984, photo of two mule deer bucks just south of Alpine, Texas, with their antlers sticking up quite happily.
I have seen bucks in velvet here in south Brewster County, Texas, at Labor Day.
Go Figure.
The size of antlers will vary with genetics, as well as diet. Diet variance can include minerals as well as "just food", and annual rainfall will affect the latter. In general, a healthy buck will have forked horns at age 1-1/2. Assuming no dietary or genetic deficiencies, the number of points and thickness of beam/length of tine will increase with age. Fullest maturity is roughly 6-1/2 to 7-1/2 years. After 7-1/2, the worn teeth lead to a decline in health and thus poorer antlers. There may be many points, but they can be spindly and of narrow spread.
Hope this helps,
Art