In 1933, the Kriegsmarine looked for a standardized shipboard reconnaissance aircraft. After a brief selection period, the Reichsluftfahrtministerium (German Air Ministry, RLM) decided on the Heinkel He 60 biplane. This was one of a line of developments of a basic biplane airframe that appeared as a number of floatplanes, trainers, and fighters. Deliveries started in a matter of months.

By 1935, it was found that the He 60's performance was lacking, and the RLM asked Heinkel to design its replacement. The result was the He 114. The first prototype was powered by the Daimler-Benz DB 600 inline engine, but it was clear that supplies of this engine would be limited, and the production versions turned to the BMW 132 radial engine instead.

The plane proved to have only slightly better performance than the He 60, and its sea-handling was poor. Rushed modifications resulted in a series of nine prototypes in an attempt to solve some of the problems, but they didn't help much. The Navy gave up, and the planes were eventually sold off to Romania, Spain and Sweden.

In October 1936, the RLM asked for a He 114 replacement. The only stipulations were that it would use the BMW 132, and they wanted prototypes in both twin-float and single-float configurations. Designs were received from Dornier, Gotha, Arado and Focke-Wulf. Heinkel declined to tender, contending that the He 114 could still be made to work.

With the exception of the Arado design, they were all conventional biplanes. That gave the Arado better performance than any of the others, and the RLM ordered four prototypes. The RLM was also rather conservative by nature, so they also ordered two of the Focke-Wulf Fw 62 design as a backup. It quickly became clear that the Arado would work effectively, and only four prototypes of the Fw 62 were built.

The Ar 196 prototypes were all delivered in summer 1937, V1 (which flew in May) and V2 with twin floats as A models, and V3 and V4 on a single float as B models. Both versions demonstrated excellent water handling, and there seemed to be little to decide one over the other. Since there was a possibility of the smaller outrigger floats on the B models "digging in", the twin-float A model was ordered into production. A single additional prototype, V5, was produced in November 1938 to test final changes.

10 A-0s were delivered in November and December 1938, with a single 7.92 mm (.312 in) MG 15 machine gun in the rear seat for defense. Five similarly equipped B-0s were also delivered to land-based squadrons. This was followed by 20 A-1 production models starting in June 1939, enough to equip the surface fleet.

Starting in November production switched to the heavier land-based A-2 model. It added shackles for two 50 kg (110 lb) bombs, two 20 mm MG FF cannons in the wings, and a 7.92 mm MG 17 machine gun in the cowling. The A-4 replaced it in December 1940, strengthening the airframe, adding another radio, and switching props to a VDM model. The apparently mis-numbered A-3 replaced the A-4, with additional strengthening of the airframe. The final production version was the A-5 from 1943, which changed radios and cockpit instruments, and switched the rear gun to the much-improved MG 81Z. In all versions, 541 Ar 196s (526 production models) were built before production ended in August 1944, about 100 of these from SNCA and Fokker plants.

The Ar 196C was a proposed aerodynamically-refined version. The Ar 196C project was cancelled in 1941.

Sources:
Gunston, Bill & Wood, Tony - Hitler's Luftwaffe, 1977, Salamander Books Ltd., London.
Wikipedia - Ar 196