TESTS ON DATA SET 1 Ordinary logistic regression lambda = 0 estimates: -0.1665 0.5376 -0.4985 error rate: 0.23833 sq. error: 0.17497 lambda = 0.1 estimates: -0.1663 0.5375 -0.4983 error rate: 0.238 sq. error: 0.17497 lambda = 0.1778279 estimates: -0.1662 0.5374 -0.4982 error rate: 0.238 sq. error: 0.17497 lambda = 0.3162278 estimates: -0.1661 0.5372 -0.4979 error rate: 0.238 sq. error: 0.17497 lambda = 0.5623413 estimates: -0.1658 0.5369 -0.4975 error rate: 0.23767 sq. error: 0.17498 lambda = 1 estimates: -0.1652 0.5362 -0.4967 error rate: 0.238 sq. error: 0.17499 lambda = 1.778279 estimates: -0.1643 0.5352 -0.4953 error rate: 0.238 sq. error: 0.17501 lambda = 3.162278 estimates: -0.1626 0.5333 -0.4929 error rate: 0.23767 sq. error: 0.17504 lambda = 5.623413 estimates: -0.1596 0.53 -0.4886 error rate: 0.23767 sq. error: 0.1751 lambda = 10 estimates: -0.1545 0.5242 -0.4813 error rate: 0.239 sq. error: 0.1752 lambda = 17.78279 estimates: -0.1458 0.5146 -0.469 error rate: 0.23733 sq. error: 0.17541 lambda = 31.62278 estimates: -0.1314 0.4988 -0.449 error rate: 0.23733 sq. error: 0.17579 lambda = 56.23413 estimates: -0.1086 0.4743 -0.4185 error rate: 0.237 sq. error: 0.17653 lambda = 100 estimates: -0.0746 0.4387 -0.3754 error rate: 0.23633 sq. error: 0.17794 lambda = 177.8279 estimates: -0.0274 0.391 -0.3203 error rate: 0.23767 sq. error: 0.1805 lambda = 316.2278 estimates: 0.0333 0.3328 -0.2573 error rate: 0.241 sq. error: 0.18481 lambda = 562.3413 estimates: 0.105 0.2678 -0.1935 error rate: 0.25467 sq. error: 0.19131 lambda = 1000 estimates: 0.1819 0.2021 -0.136 error rate: 0.28867 sq. error: 0.19984 Bounded logistic regression lambda = 0 estimates: -0.8152 -1.7386 0.0475 1.4679 -1.311 error rate: 0.23867 sq. error: 0.17073 lambda = 0.1 estimates: -0.8213 -1.7625 0.0504 1.4517 -1.2958 error rate: 0.23867 sq. error: 0.17071 lambda = 0.1778279 estimates: -0.8258 -1.7806 0.0525 1.4397 -1.2847 error rate: 0.23867 sq. error: 0.1707 lambda = 0.3162278 estimates: -0.8336 -1.812 0.0561 1.4198 -1.266 error rate: 0.23867 sq. error: 0.17068 lambda = 0.5623413 estimates: -0.8465 -1.8656 0.0621 1.3877 -1.2359 error rate: 0.23867 sq. error: 0.17064 lambda = 1 estimates: -0.8671 -1.9554 0.0713 1.339 -1.1904 error rate: 0.23867 sq. error: 0.1706 lambda = 1.778279 estimates: -0.8988 -2.1031 0.0847 1.2704 -1.1262 error rate: 0.23867 sq. error: 0.17056 lambda = 3.162278 estimates: -0.9452 -2.3418 0.1017 1.1819 -1.0433 error rate: 0.23833 sq. error: 0.17053 lambda = 5.623413 estimates: -1.0105 -2.7172 0.1188 1.0781 -0.9465 error rate: 0.239 sq. error: 0.17057 lambda = 10 estimates: -1.1009 -3.2707 0.1273 0.9664 -0.8434 error rate: 0.238 sq. error: 0.17074 lambda = 17.78279 estimates: -1.2317 -4.1651 0.1225 0.848 -0.7352 error rate: 0.23833 sq. error: 0.17117 lambda = 31.62278 estimates: -1.435 -14.9442 0.103 0.7246 -0.6233 error rate: 0.239 sq. error: 0.17211 lambda = 56.23413 estimates: -1.7828 -18.5325 0.0578 0.6038 -0.516 error rate: 0.23667 sq. error: 0.17387 lambda = 100 estimates: -15.4164 -13.7145 -0.0746 0.4387 -0.3754 error rate: 0.23633 sq. error: 0.17794 lambda = 177.8279 estimates: -19.7569 -13.7709 -0.0274 0.391 -0.3203 error rate: 0.23767 sq. error: 0.1805 lambda = 316.2278 estimates: -34.8751 -18.105 0.0333 0.3328 -0.2573 error rate: 0.241 sq. error: 0.18481 lambda = 562.3413 estimates: -28.4284 -13.9447 0.105 0.2678 -0.1935 error rate: 0.25467 sq. error: 0.19131 lambda = 1000 estimates: -36.4925 -16.773 0.1819 0.2021 -0.136 error rate: 0.28867 sq. error: 0.19984 TESTS ON DATA SET 2 Ordinary logistic regression lambda = 0 estimates: 0.2594 -4.4587 -7.5043 3.5327 3.5944 -3.1661 1.0798 3.0512 1.9038 -0.6799 0.8625 0.7934 0.8486 -0.2462 -0.0237 0.1313 0.8596 -0.4244 -0.1283 -0.1638 0.6153 error rate: 0.23778 sq. error: 0.17286 lambda = 0.1 estimates: 0.5553 -1.1213 -0.8828 1.1623 0.8237 -1.5449 -0.0117 0.8533 -0.1769 -0.5811 0.2626 0.3941 0.6285 -0.1558 0.069 0.2921 0.5319 -0.2481 -0.1441 -0.0674 0.2413 error rate: 0.22 sq. error: 0.14964 lambda = 0.1778279 estimates: 0.5411 -0.794 -0.5112 0.7556 0.5915 -1.1501 -0.0087 0.5492 -0.2144 -0.5164 0.1686 0.331 0.6019 -0.1508 0.0788 0.2876 0.4844 -0.2469 -0.1512 -0.0613 0.218 error rate: 0.22 sq. error: 0.14791 lambda = 0.3162278 estimates: 0.51 -0.5618 -0.2813 0.4274 0.4078 -0.8157 0.0059 0.3147 -0.1832 -0.444 0.0936 0.2713 0.5688 -0.15 0.0864 0.2758 0.4379 -0.2482 -0.1539 -0.0573 0.1998 error rate: 0.21556 sq. error: 0.1466 lambda = 0.5623413 estimates: 0.4627 -0.4051 -0.1432 0.1942 0.2742 -0.5658 0.0198 0.15 -0.1261 -0.3739 0.0439 0.2137 0.5236 -0.1516 0.0924 0.259 0.3897 -0.2464 -0.1486 -0.0539 0.1825 error rate: 0.21111 sq. error: 0.14538 lambda = 1 estimates: 0.3967 -0.3015 -0.0607 0.0471 0.1868 -0.3948 0.03 0.0436 -0.0705 -0.3109 0.0189 0.1561 0.4635 -0.1534 0.0968 0.2376 0.3365 -0.2372 -0.1337 -0.0499 0.1637 error rate: 0.21222 sq. error: 0.14401 lambda = 1.778279 estimates: 0.3102 -0.2328 -0.0109 -0.0357 0.1348 -0.2837 0.0378 -0.0201 -0.0266 -0.2561 0.0129 0.0992 0.391 -0.1535 0.0993 0.211 0.277 -0.2184 -0.1108 -0.0447 0.1426 error rate: 0.20111 sq. error: 0.14242 lambda = 3.162278 estimates: 0.205 -0.1862 0.0193 -0.0764 0.1059 -0.2126 0.044 -0.0557 0.0051 -0.21 0.018 0.047 0.3136 -0.1501 0.0989 0.179 0.2141 -0.1899 -0.0837 -0.0381 0.1188 error rate: 0.2 sq. error: 0.14071 lambda = 5.623413 estimates: 0.0858 -0.1534 0.0372 -0.0926 0.0895 -0.1665 0.0486 -0.0739 0.0264 -0.1728 0.0274 0.004 0.2407 -0.1418 0.0944 0.1429 0.1544 -0.1543 -0.0572 -0.0303 0.0931 error rate: 0.2 sq. error: 0.13909 lambda = 10 estimates: -0.0426 -0.1292 0.0461 -0.0955 0.0783 -0.1354 0.0507 -0.0812 0.039 -0.1435 0.0359 -0.0269 0.1788 -0.1286 0.0854 0.1064 0.104 -0.1166 -0.035 -0.0218 0.0672 error rate: 0.19889 sq. error: 0.1378 lambda = 17.78279 estimates: -0.1788 -0.1099 0.0483 -0.0911 0.0684 -0.1125 0.0496 -0.0812 0.0442 -0.1201 0.0404 -0.0457 0.1302 -0.1116 0.0728 0.0738 0.0663 -0.0822 -0.019 -0.0138 0.0443 error rate: 0.20333 sq. error: 0.13709 lambda = 31.62278 estimates: -0.3252 -0.0929 0.0452 -0.0824 0.0579 -0.0937 0.0453 -0.0758 0.0433 -0.1001 0.0401 -0.0537 0.0933 -0.0932 0.0584 0.0482 0.0405 -0.0549 -0.009 -0.0077 0.0267 error rate: 0.20556 sq. error: 0.13726 lambda = 56.23413 estimates: -0.486 -0.0767 0.0388 -0.0708 0.0468 -0.0767 0.0384 -0.0666 0.0381 -0.0818 0.0357 -0.0533 0.0658 -0.0751 0.0444 0.0301 0.0241 -0.0353 -0.0035 -0.0037 0.0149 error rate: 0.20778 sq. error: 0.13868 lambda = 100 estimates: -0.6631 -0.0609 0.0306 -0.0577 0.0357 -0.0606 0.0302 -0.0549 0.0306 -0.0645 0.0289 -0.0472 0.0453 -0.0585 0.0321 0.0183 0.0141 -0.0221 -9e-04 -0.0016 0.0079 error rate: 0.21333 sq. error: 0.14167 lambda = 177.8279 estimates: -0.8526 -0.0459 0.0224 -0.0442 0.0256 -0.0456 0.0221 -0.0423 0.0226 -0.0484 0.0215 -0.038 0.0303 -0.0436 0.0221 0.0109 0.0081 -0.0136 1e-04 -6e-04 0.004 error rate: 0.21444 sq. error: 0.14628 lambda = 316.2278 estimates: -1.0411 -0.0326 0.0153 -0.0316 0.0173 -0.0323 0.0151 -0.0304 0.0155 -0.0342 0.0148 -0.0279 0.0195 -0.0308 0.0144 0.0064 0.0046 -0.0083 4e-04 -2e-04 0.0021 error rate: 0.21556 sq. error: 0.15192 lambda = 562.3413 estimates: -1.2096 -0.0217 0.0098 -0.0211 0.011 -0.0215 0.0097 -0.0204 0.01 -0.0227 0.0096 -0.019 0.0121 -0.0205 0.009 0.0037 0.0026 -0.0049 4e-04 -1e-04 0.0011 error rate: 0.21556 sq. error: 0.15758 lambda = 1000 estimates: -1.3426 -0.0136 0.006 -0.0133 0.0067 -0.0135 0.0059 -0.0129 0.0061 -0.0142 0.0059 -0.0121 0.0072 -0.0129 0.0055 0.0021 0.0015 -0.0029 3e-04 0 6e-04 error rate: 0.21556 sq. error: 0.16229 Bounded logistic regression lambda = 0 estimates: -0.2232 -1.2669 -12.0778 -40.6616 13.2845 -40.2048 13.7603 -40.4596 13.8919 -40.6266 14.0136 -41.9948 13.3438 -36.1217 15.5303 -42.1993 13.2841 13.2228 12.628 -1.3945 1.3008 -12.0978 -12.343 error rate: 0.21556 sq. error: 0.17331 lambda = 0.1 estimates: -20.8331 -2.0435 0.3446 -0.3036 0.5713 0.2074 0.3337 -0.1529 0.2286 -0.2288 0.849 -2.3021 0.6334 1.259 1.9932 -1.3821 0.0201 -0.1712 1.6162 -0.4376 -0.3213 -0.4494 0.873 error rate: 0.20333 sq. error: 0.16852 lambda = 0.1778279 estimates: -20.355 -2.0418 0.2492 -0.2543 0.4834 0.1042 0.2913 -0.1579 0.2208 -0.1926 0.6877 -1.8384 0.5459 1.054 1.6507 -1.1585 0.025 -0.1375 1.3458 -0.3508 -0.3 -0.3747 0.7166 error rate: 0.20222 sq. error: 0.16527 lambda = 0.3162278 estimates: -24.9217 -2.0804 0.6691 -0.329 0.173 -0.0919 0.1285 -0.1568 0.2461 -0.4417 0.7276 -0.7482 -0.0537 0.8042 1.2119 -1.1029 -0.502 0.8964 0.8338 -0.0617 -0.3133 0.1538 0.4681 error rate: 0.21889 sq. error: 0.1694 lambda = 0.5623413 estimates: -21.1657 -2.1086 0.1076 -0.188 0.2858 -0.0011 0.1972 -0.1893 0.167 -0.1243 0.4203 -0.9901 0.3597 0.6697 1.0121 -0.7391 -0.0161 -0.0184 0.8788 -0.1733 -0.2177 -0.1843 0.4525 error rate: 0.20222 sq. error: 0.15666 lambda = 1 estimates: -24.2032 -2.3318 0.1228 -0.1713 0.1787 -0.0129 0.1514 -0.2168 0.1321 -0.0853 0.3129 -0.621 0.2298 0.4907 0.7361 -0.5717 -0.0584 0.0659 0.7095 -0.0946 -0.166 -0.0666 0.3662 error rate: 0.19778 sq. error: 0.15184 lambda = 1.778279 estimates: -24.8164 -23.3392 0.3102 -0.2328 -0.0109 -0.0357 0.1348 -0.2837 0.0378 -0.0201 -0.0266 -0.2561 0.0129 0.0992 0.391 -0.1535 0.0993 0.211 0.277 -0.2184 -0.1108 -0.0447 0.1426 error rate: 0.20111 sq. error: 0.14242 lambda = 3.162278 estimates: -22.3439 -54.8437 0.205 -0.1862 0.0193 -0.0764 0.1059 -0.2126 0.044 -0.0557 0.0051 -0.21 0.018 0.047 0.3136 -0.1501 0.0989 0.179 0.2141 -0.1899 -0.0837 -0.0381 0.1188 error rate: 0.2 sq. error: 0.14071 lambda = 5.623413 estimates: -23.0601 -99.2004 0.0858 -0.1534 0.0372 -0.0926 0.0895 -0.1665 0.0486 -0.0739 0.0264 -0.1728 0.0274 0.004 0.2407 -0.1418 0.0944 0.1429 0.1544 -0.1543 -0.0572 -0.0303 0.0931 error rate: 0.2 sq. error: 0.13909 lambda = 10 estimates: -26.9557 -197.1667 -0.0426 -0.1292 0.0461 -0.0955 0.0783 -0.1354 0.0507 -0.0812 0.039 -0.1435 0.0359 -0.0269 0.1788 -0.1286 0.0854 0.1064 0.104 -0.1166 -0.035 -0.0218 0.0672 error rate: 0.19889 sq. error: 0.1378 lambda = 17.78279 estimates: -23.8133 -210.5675 -0.1788 -0.1099 0.0483 -0.0911 0.0684 -0.1125 0.0496 -0.0812 0.0442 -0.1201 0.0404 -0.0457 0.1302 -0.1116 0.0728 0.0738 0.0663 -0.0822 -0.019 -0.0138 0.0443 error rate: 0.20333 sq. error: 0.13709 lambda = 31.62278 estimates: -23.7732 -234.2301 -0.3252 -0.0929 0.0452 -0.0824 0.0579 -0.0937 0.0453 -0.0758 0.0433 -0.1001 0.0401 -0.0537 0.0933 -0.0932 0.0584 0.0482 0.0405 -0.0549 -0.009 -0.0077 0.0267 error rate: 0.20556 sq. error: 0.13726 lambda = 56.23413 estimates: -23.0893 -265.9004 -0.486 -0.0767 0.0388 -0.0708 0.0468 -0.0767 0.0384 -0.0666 0.0381 -0.0818 0.0357 -0.0533 0.0658 -0.0751 0.0444 0.0301 0.0241 -0.0353 -0.0035 -0.0037 0.0149 error rate: 0.20778 sq. error: 0.13868 lambda = 100 estimates: -22.9788 -300.9492 -0.6631 -0.0609 0.0306 -0.0577 0.0357 -0.0606 0.0302 -0.0549 0.0306 -0.0645 0.0289 -0.0472 0.0453 -0.0585 0.0321 0.0183 0.0141 -0.0221 -9e-04 -0.0016 0.0079 error rate: 0.21333 sq. error: 0.14167 lambda = 177.8279 estimates: -23.4836 -341.7209 -0.8526 -0.0459 0.0224 -0.0442 0.0256 -0.0456 0.0221 -0.0423 0.0226 -0.0484 0.0215 -0.038 0.0303 -0.0436 0.0221 0.0109 0.0081 -0.0136 1e-04 -6e-04 0.004 error rate: 0.21444 sq. error: 0.14628 lambda = 316.2278 estimates: -20.1611 -343.5575 -1.0411 -0.0326 0.0153 -0.0316 0.0173 -0.0323 0.0151 -0.0304 0.0155 -0.0342 0.0148 -0.0279 0.0195 -0.0308 0.0144 0.0064 0.0046 -0.0083 4e-04 -2e-04 0.0021 error rate: 0.21556 sq. error: 0.15192 lambda = 562.3413 estimates: -19.5753 -411.1257 -1.2096 -0.0217 0.0098 -0.0211 0.011 -0.0215 0.0097 -0.0204 0.01 -0.0227 0.0096 -0.019 0.0121 -0.0205 0.009 0.0037 0.0026 -0.0049 4e-04 -1e-04 0.0011 error rate: 0.21556 sq. error: 0.15758 lambda = 1000 estimates: -13.0964 -346.6976 -1.3426 -0.0136 0.006 -0.0133 0.0067 -0.0135 0.0059 -0.0129 0.0061 -0.0142 0.0059 -0.0121 0.0072 -0.0129 0.0055 0.0021 0.0015 -0.0029 3e-04 0 6e-04 error rate: 0.21556 sq. error: 0.16229