The first expression is understandable. 'Not more than five mistakes were found in the essay' - for simplicity, we would most likely say 'Fewer than five mistakes were found in the essay'. 'Fewer' means 'not more than'.
The second sentence is too confusing. Nobody would use this wording.