At age eighteen Baldwin commenced teaching. He spent the next four years alternating between the occupations of farmer, student and teacher. During this period he received his formal education at the Delaware Literary Institute at Franklin, NY where he pursued the study of law. Stressing the need for a proper education, Baldwin wrote, "study away, you will never regret a good education. If I had a good, intelligent son, I don’t know but I should make his young life bitter with intellectual training."

In the spring of 1861, the twenty-three year old Baldwin was commissioned a second lieutenant in Company R, 20^th New York State Militia. This company was also known as the Lexington Artillery, originally formed in 1820. Although his first term of service lasted only three months, he achieved a reputation as a proficient and energetic officer. During the regiment’s reorganization period, Baldwin was instrumental in recruiting a company from the area of Jewett, Hunter and neighboring towns. Coupled with the efforts of his college companion John Ross Horner, the unit soon numbered 100 volunteers and was designated Company K. On Sept. 16, 1861 Baldwin was mustered as Captain of the company. A "capable and dauntless officer", Baldwin participated in the battles of Norman’s Ford, Warrenton Springs, Run, Chantilly and South Mountain.

Almost one year to the day of his muster, Sept. 17, 1861, at the Battle of Antietam, Baldwin was wounded in the knee by a bursting shell which "cut through three thickness of rubber coat which he wore at the time", creating a slight wound which produced a lame knee. After a short leave Baldwin returned to duty.

In the early afternoon of July 1, 1863, Capt. Baldwin and the men of Company K advanced 500 yards in front of the main Federal line. Their objective was to silence Confederate sharpshooters positioned on the Emanuel Harman farm. This gallant band of approximately 40 men pushed across an open field, down a ravine, splashed across Willoughby Run, and struggled up a steep embankment to roust the North Carolinians within the span of 15 minutes.

The Confederates quickly returned, engaging Baldwin’s forlorn Company K. Lt. John Young of Catskill, quickly entreated Col. Theodore Gates for reinforcements. Capt. William Cunningham’s Company G was dispatched with alacrity. Company G quickly arrived at the farm and the two plucky Greene County companies soon found themselves holding the entire Confederate right flank in check for nearly an hour. At about 1 pm the Confederates launched a massive assault which forced Baldwin and Cunningham to fall back. They reunited with their comrades early in the evening on Cemetery Hill. They found the 20^th to be a mere shell of its former self, having sustained heavy casualties in the day’s fighting.

On Friday, July 3, 1863, the men of the 20^th Militia found themselves near the center of the Union line facing the Confederate onslaught known as "Pickett’s Charge". In the ensuing melee Capt. Baldwin was "struck by a piece of shell which nearly tore his leg from his body", causing a mortal wound. Baldwin was carried to a nearby field hospital by two stalwart soldiers to whom he "expressed his thanks, saying he feared he never could repay them for their kindness". Amid the horrifying hospital atmosphere, where surgeons labored feverishly, Baldwin bled to death. From available accounts, "no friend or acquaintance spoke to him" prior to his demise. Tragically, like many others, he died alone, far from family and friends.

The day following his death, July 4, 1863, Capt. Baldwin was buried on the battlefield. On the 11^th his brother Horace, accompanied by William Perry, left Gettysburg and brought the remains home for burial. On Sunday, July 19, 1863, "in accordance with his own desire to repose by the side of his sainted mother", the body was interred in the family plot in Jewett Heights Cemetery.