Webwork compares functions by plugging in some points and seeing if they match. This error means that Webwork failed when trying to compare the student's function/equation to the correct function/equation. This occurs when the numbers it tries to plug in to the correct function are not in the domain. It can also occur due to computational limitations of WeBWorK's Perl-based arithmetic engine:
To fix this, you can explicitly tell Webwork which points to plug in to the functions. Replace, e.g.,
fun_cmp("[correct answer]")
by
fun_cmp("[correct answer]", limits => [a,b] )
where [a,b] is the interval from which Webwork should pick test points (note that the default test interval for each interval is [-2,2]), or by
fun_cmp("[correct answer]", test_points=>[a,b,c,d,e] )
where a, b, c, d, and e are the specific points which Webwork should test. See http://webwork.maa.org/wiki/FormulaTestPoints for more details.