Ilfrey entered the US Army Air Corps as an Aviation Cadet in April of 1941. He graduated with the first wartime class of pilots at Luke Field, Arizona, on 12 December, 1941, (Class 41-I) and was assigned to the famed 94th Pursuit "Hat In The Ring" Squadron, 1st Pursuit Group. His first assignment found him flying P-38D and E's in defense of the southern California coastline.

In the spring of 1942, the 1st Fighter Group (as they were now designated,) were equipped with new P-38F's and were ordered to Dow Field, Maine to prepare for the "Bolero Mission," the first mass flight of fighter planes and bombers to England. "We called it the 'Guinea Pig Mission', cause that is what we were, guinea pigs. " Ilfrey later stated. On July 4th, 1942 the 94th squadron took off on the first leg of the mission from Presque Isle, Maine. On July 26th, most of the squadron landed at Kirton, in Lindsey, Lincolnshire, England. They were stationed there with the Polish 303rd Koscuisko Squadron, who taught them many of the tricks of the trade. On September 1st, the 1st Fighter Group made the first all American fighter sweep over northern France from Beurnmouth, along the coastline of the English Channel.

On November 15th the group took off from Chivenor, in the Lands End area of England, on "Operation Torch," the invasion of North Africa. It was during this mission that the first of many noteworthy happenings in Ilfrey's career occurred. Shortly after take off, Ilfrey lost a belly tank, which meant he would not have enough fuel to complete the mission. He calculated that he had enough to make an emergency landing in Gibraltar, but his calculations proved incorrect and he was forced to land in Lisbon, Portugal.

Jack soon put all of his training into action over North Africa. On the 29th of November he shared credit for downing a Messerschmitt Bf-110 near Gabes Airdrome, in Tunisia. On December 2nd he downed two Messerschmitt Me-109s over Gabes Airdrome, Tunisia, and on December 26th he shot down two Focke Wulf Fw-190s five miles west of Bizerte. On January 11, 1943 he damaged a Me-109 five miles north of Gabes Airdrome and on March 3rd he downed another Me-109 near El Aounia, Tunisia, making him one of the first pilots (many believe he was the first) to become an ace while flying a P-38. The action continued when, on the following day, he damaged another Me-109 over Bizerte Harbor.

He received a commendation for his actions from the Chief of Staff, US Army, on February 3rd, 1943. After a total of five and a half confirmed air to air victories and two confirmed damaged enemy aircraft, 208 combat hours and 72 missions, Jack was relieved of combat duty and reassigned to the states as a flight instructor in P-38s and P-47 Thunderbolts at the replacement training unit based in Santa Ana, California.

Ilfrey returned to the E.T.O. in April of 1944, as the Operations Officer of the 79th Fighter Squadron, 20th Fighter Group, based in King's Cliffe, NorthamPTOnshire, England. This group was also equipped with P-38s. On the 27th of September he became the Squadron's Commander.

Soon after his arrival he was back in action. On May 24th, during a mission to Berlin, he was credited with downing two Messerschmitt Me-109s, one of which had actually collided with Ilfrey's Lightning and had sheared off nearly four and a half to five feet of Ilfrey's right wing. The enemy pilot did not recover from the subsequent spin, but Ilfrey's skill kept him from joining in the fate of his foe. Jack managed to bring the P-38 back home only to discover many of his squadron-mates had given up on him making it back.

As D-Day approached, and Invasion stripes were added to their aircraft, the 79th Fighter Squadron readied itself for some intense action. Ilfrey flew three patrols over Normandy on June 6th (D-Day), as the largest invasion force to ever cross the English Channel fought below him on the beaches of Normandy. The Luftwaffe did not make an appearance in the skies over Normandy that day so the men of the 20th Fighter Group had the chance to watch the invasion unfold beneath them.

On June 12th, after successfully dive bombing a railway bridge over the Loire River, near Angers, France, Jack was shot down while strafing a nearby train. He was over 200 miles behind the front lines, but with the help of the French civilians he managed to evade capture dressed as a French farmer Jacques Robert' and was back in England within six days. Despite the official doctrine which sent all pilots who evaded back to the United States he somehow managed to side-step the rules and continued flying in combat.

In late July, the 20th Fighter Group converted to the P-51D Mustang and within months were involved in "Operation Frantic VI", a shuttle mission to Russia, Italy and back to England. It was around this time that Ilfrey was promoted to Major, however the celebration that followed said promotion led to his being busted back to 2nd Lieutenant for multiple infractions. He remained in command of the squadron, however, and his claim to fame...or infamy... (as he was fond of saying) is he was the only 2nd Lieutenant to ever lead a Combat Squadron during the war. He soon returned to the rank of Captain however, as Gen. Doolittle, now in command of the 8th Air Force, was once again persuaded to overlook the infractions by 20th Fighter Group C.O., Colonel Cy Wilson.

The remainder of Ilfrey's second tour of duty in the E.T.O. remained fairly uneventful until the 20th of November, when he landed behind enemy lines, in Maastrick, Holland and successfully retrieved his downed wingman, Lieutenant Duane Kelso. Ilfrey and his wingman squeezed into the cockpit of the P-51 (designed for only one person,) and flew a short trip to Brussels where they landed safely.

On December 9, 1944, after a total of 70 missions, 320 hours of combat flying and two enemy aircraft downed, Ilfrey was reassigned to the US, where he became a Troop Commander at McChord Air Force Base. He left the USAAF with the final rank of Major in December of 1945. In two tours of duty Ilfrey completed a total of 142 missions, flew 528 combat hours, downed an officially confirmed seven and a half enemy aircraft (all air victories), damaged two more and destroyed two enemy trains on the ground. In December, 1945 he left the service, purchased a Beechcraft Model 18 commercial light transport and became a pilot for a Dallas, Texas-based construction firm. From 1949 through 1951 Ilfrey also trained foreign pilots in Hondo, Texas but, as his eyesight began fading, he began to seek out work that would keep him closer to the ground.

Ilfrey retired after 30 years as a bank officer at Alamo National Bank, in San Antonio, Texas. In addition to being the Historian, and a past President, of The 20th Fighter Group Association he was the Editor of the group's newsletter, Kings Cliffe Remembered, throughout it's entire run of over fifty issues. He was active in the 8th Air Force Historical Society, The 1st Fighter Group Association, The American Fighter Aces Association, The Commemorative Air Force and the P-38 National Association. On October 6, 1998 he was inducted into the Commemorative Air Force's American Combat Airman Hall of Fame. He was also an invaluable resource to aviation enthusiasts, researchers, historians and model-builders alike and was widely known for his generosity to those interested in preserving aviation history.

In 1946, he penned his acclaimed autobiography, Happy Jack's Go Buggy. The foreward to this book was written by none other than aviation legend Gen. Eddie Rickenbacker. It is considered by many one of the best glimpses into the life of an USAAF pilot ever written and received praise from the likes of General James Doolittle, General William H. Simpson, Senator Barry Goldwater and General Ira Eaker, in addition to noted aviation authors Roger Freeman and Len Deighton. It was finally published in 1979 and reprinted in an expanded edition in 1998.

Ilfrey resided in San Antonio, Texas during the final years of his life. He passed away on October 15, 2004, at Nix Hospital, after several months of declining health. He was buried next to his parents in Forest Park Lawndale Cemetery, Houston, Texas on October 19th.