Serial number disclosure is a risk, but if a security vendor fails that test, well...
FortiGate-61E (19:30-06.08.2016)
Ver:05000009
Serial number: FGT61E4Q16000530
CPU: 1000MHz
Total RAM: 2 GB
Initializing boot device...
Initializing MAC... nplite#0
Please wait for OS to boot, or press any key to display configuration menu......
Booting OS...
Reading boot image... 2702032 bytes.
Initializing firewall...
System is starting...
FGT61E4Q16000530 login: admin
Password:
Welcome !
FGT61E4Q16000530 # get hardware status
Model name: FortiGate-61E
ASIC version: SOC3
ASIC SRAM: 64M
CPU: ARMv7
Number of CPUs: 4
RAM: 1866 MB
EMMC: 3662 MB(MLC) /dev/mmcblk0
Hard disk: 122104 MB /dev/sda
USB Flash: not available
Network Card chipset: FortiASIC NP6LITE Adapter (rev.)
FGT61E4Q16000530 # get sys status
Version: FortiGate-61E v5.4.1,build5495,160714 (GA)
Virus-DB: 1.00123(2015-12-11 13:18)
Extended DB: 1.00000(2012-10-17 15:46)
IPS-DB: 6.00741(2015-12-01 02:30)
IPS-ETDB: 0.00000(2001-01-01 00:00)
Serial-Number: FGT61E4Q16000530
IPS Malicious URL Database: 1.00001(2015-01-01 01:01)
Botnet DB: 1.00000(2012-05-28 22:51)
BIOS version: 05000009
System Part-Number: P18817-01
Log hard disk: Available
Hostname: FGT61E4Q16000530
Operation Mode: NAT
Current virtual domain: root
Max number of virtual domains: 10
Virtual domains status: 1 in NAT mode, 0 in TP mode
Virtual domain configuration: disable
FIPS-CC mode: disable
Current HA mode: standalone
Branch point: 1064
Release Version Information: GA
System time: Sat Nov 12 08:43:23 2016
FGT61E4Q16000530 # diag hardware sysinfo cpu
Processor: ARMv7 Processor rev 1 (v7l)
processor: 0
BogoMIPS: 2007.04
processor: 1
BogoMIPS: 2007.04
processor: 2
BogoMIPS: 2007.04
processor: 3
BogoMIPS: 2007.04
Features: swp half thumb fastmult vfp edsp thumbee vfpv3 vfpv3d16 tls
CPU implementer: 0x41
CPU architecture: 7
CPU variant: 0x4
CPU part: 0xc09
CPU revision: 1
Hardware: FSoC3_ASIC
Revision: 0000
Serial: 0000000000000000
FGT61E4Q16000530 # diag hardware sysinfo cpumem
MemTotal: 1911672 kB
MemFree: 1393712 kB
Buffers: 4568 kB
Cached: 104772 kB
SwapCached: 0 kB
Active: 116148 kB
Inactive: 79192 kB
Active(anon): 94172 kB
Inactive(anon): 25436 kB
Active(file): 21976 kB
Inactive(file): 53756 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 86000 kB
Mapped: 35276 kB
Shmem: 33608 kB
Slab: 19708 kB
SReclaimable: 7200 kB
SUnreclaim: 12508 kB
KernelStack: 808 kB
PageTables: 6380 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 955836 kB
Committed_AS: 15610676 kB
VmallocTotal: 663552 kB
VmallocUsed: 33752 kB
VmallocChunk: 511456 kB
FGT61E4Q16000530 # diag hardware deviceinfo disk
Disk SYSTEM(boot) ref: 3.6GB type: EMMC [EMMC] dev: /dev/mmcblk0
partition ref: 247.0MB, 206.0MB free mounted: Y label: dev: /dev/mmcblk0p1(boot) start: 0
partition ref: 256.0MB, 256.0MB free mounted: N label: dev: /dev/mmcblk0p2(boot) start: 0
partition ref: 3 2.9GB, 2.9GB free mounted: Y label: dev: /dev/mmcblk0p3 start: 0
Disk Internal ref: 258 119.2GB type: SSD [ATA LITEON CV1-8B128] dev: /dev/sda
partition ref: 259 117.4GB, 117.2GB free mounted: Y label: LOGUSEDX929BA67B dev: /dev/sda1 start: 2048
Total available disks: 2
Max SSD disks: 1 Available storage disks: 1
Note that these apply to one or more of FortiOS 5.4.1/2/3/4 on the 61E. I'm not trying to be comprehensive here.
- GUI FortiView "All Sessions" log display did not function initially (5.4.1) - I could only get data under "now". It works now, after a reset to factory defaults, two power outages, reboot and fsck, and reconfiguration with no UTM config. Something in there fixed it, apparently.
- It's easy to screw up the GUI FortiView display if you get too impatient and clicky while it's trying to display data (the poor 1GHz ARM and my old 2.2GHz K10 workstation combine for some pretty long load times). Could be a browser issue (Firefox) - I suppose I could poke it in Edge, if I cared enough.
- I could not convince nTurbo to work (offload to the NP6Lite with UTM features enabled) under 5.4.1; updating to 5.4.2 appears to have helped. (I threw some sample session displays below.) (Update: The definition of "nTurbo" is a bit vague. Some documentation explicitly indicates that the CP is used for Application Control, but without counters the actual operation of the CP seems impossible to determine.)
- I tried two configurations that did not work (for me):
- Using the vlink interfaces to communicate between the root NAT/route VDOM and the transparent VDOM. Connecting two physical interfaces works fine.
- Proxy ARP. I've used it to pseudo-route on bridge-ish (consumer Internet) services off and on for years using routing equipment, but I've had no luck with firewalls (Netscreen, Fortinet).
- Counters! Grrrrr. Coming from a 100D (no NPU), the 61E is a bit disappointing. The NP6 logging config is not present; the device appears to have essentially three levels of logging (referring to characteristics with all logging options enabled); I grabbed a sample default local report for each (one day, similar traffic patterns: speed tests, YouTube, other junk):
- Minimal: Offload to NP6Lite, no UTM. This appears to log only the session establishment (i.e. the packet[s] processed by the CPU), so the session byte counts are not captured.
- Indeterminate: Offload to NP6Lite (Application Control, no IPS). This captures... some byte counters, but I didn't see a pattern offhand.
- Normal: No offload to NP6Lite. This seems to keep accurate byte counters.
One of these days I'll settle on appropriate configurations (at a policy level), examine these conditions in more detail, and create some appropriate custom reports.
- Policy matching is a bit wonky. I use explicit denies, as I like to see match counters. The Fortigate still matches on the implicit denies, with the source and/or destination interface listed as "unknown-0". Some of these may be attributable to incorrectly closed TCP sessions (bug?); others are broadcast traffic, mostly DHCP (which shouldn't be present on the subnet in question, but that's another issue). As always, better (more complete/detailed) logging would be nice (e.g. source MAC for the DHCP).
- More Fortiview display randomness: Some sessions are identified by "Application" and others "unknown". Some source/destinations display DNS name, some don't, often in adjacent lines - when the name is obviously available (in "Log Details").
- The dreaded DNS loop! My DNS servers are inside of the firewall; when I configured DNS lookup for logging ("config log setting" → "set resolve-ip enable"), I got endless query loops on error responses. If the queries were confined to the local network (cached on the servers), this would be annoying, but I know of only a few DNS servers that will (n)cache "SERVFAIL" and none that will cache "REFUSED". So my servers were happily continuously querying the remotes, exactly as they were being asked to do, at a fairly high rate (network and/or response-rate limited) - a bit more than annoying. (Short sequence: Server sends query, firewall sees packets go by and queries server, repeat. Since the responses are errors, neither the server nor the firewall will cache them, and neither implements an appropriate rate-limit. Cheesy.) I don't see a workaround, so I disabled the feature.
- Device detection seems to be getting worse (currently 5.4.4). My TiVo is now unknown and my Linux server is Windows (this one cycles occasionally).
- I'm seeing how far I can push the log database. I have both VDOMs set at 180 days - one log is over 1GB (little enough on a 128GB SSD) after about 60 days. Seems to work so far (searches are a bit slow, naturally).
Update to 5.6.0...
- GUI Dashboard - occasionally switching from global to VDOM display retains global "Session" widget, which displays no data under the VDOM (Edge).
- GUI Dashboard, VDOM, Main - widgets are (apparently) common for all VDOMs, so the "Interface Bandwidth" widget is blank for any interfaces not in the currently selected VDOM. Clunky. A custom dashboard might allow for a more appropriate choice of widgets for the VDOM, but not really worth monkeying with.
- Policy config: Apparently the new VDOM "NGFW Mode" = "Policy-based" (old style is "Profile-based") is required to configure some policy UTM parameters in the GUI, namely Application Control and SSL Inspection profiles. I don't care to test the new mode at the moment (it doesn't offer functionality that I use, and the operational ramifications of the switch are unclear).
- GUI Dashboard, Global - "Session" widget always reports 0% nTurbo (UTM, Application Control enabled, NP6Lite offload enabled and apparently working). As I noted above, nTurbo operation is vague. For instance, would a session log that lacks "no_ofld_reason: denied-by-nturbo" indicate nTurbo operation? (I don't care to fire up the IPS to test.)
This is just my initial template, but it illustrates a couple of elements I favor operationally: Disabling the virtual-switch (to allow filtering across all ports) and setting the inspection mode to "flow" (for NP/SP offload and nTurbo).
Eliminate virtual-switch (configure for individual ports):
FGT61E4Q16000530 # config firewall policy
FGT61E4Q16000530 (policy) # delete 1
FGT61E4Q16000530 (policy) # end
FGT61E4Q16000530 # config system dhcp server
FGT61E4Q16000530 (server) # delete 1
FGT61E4Q16000530 (server) # end
FGT61E4Q16000530 # config system virtual-switch
FGT61E4Q16000530 (virtual-switch) # delete internal
FGT61E4Q16000530 (virtual-switch) # end
Set inspection mode to "flow" (to allow NP/SP offload):
FGT61E4Q16000530 # config system settings
FGT61E4Q16000530 (settings) # set inspection-mode flow
FGT61E4Q16000530 (settings) # end
Fire up VDOMs (I use two, one NAT and one transparent):
FGT61E4Q16000530 # config system global
FGT61E4Q16000530 (global) # set vdom-admin enable
FGT61E4Q16000530 (global) # end
You will be logged out for the operation to take effect
Do you want to continue? (y/n)y
exit
FGT61E4Q16000530 # config vdom
FGT61E4Q16000530 (vdom) # edit tdmz
current vf=tdmz:3
FGT61E4Q16000530 (tdmz) # config system settings
FGT61E4Q16000530 (settings) # set opmode transparent
FGT61E4Q16000530 (settings) # set inspection-mode flow
FGT61E4Q16000530 (settings) # set manageip [I use an oddball IP; some may use an actual reachable IP]
FGT61E4Q16000530 (settings) # end
Changing to TP mode
Verify a few default settings (default to highest performance; these can be configured at many levels for other purposes):
np-accel-mode = basic
cp-accel-mode = advanced
These samples span FortiOS 5.4.1 - 5.4.4, and several configurations.
Typical DNS sessions:
session info: proto=17 proto_state=01 duration=20 expire=159 timeout=0 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=4
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=log may_dirty ndr npu none
statistic(bytes/packets/allow_err): org=124/1/1 reply=437/1/1 tuples=3
tx speed(Bps/kbps): 6/0 rx speed(Bps/kbps): 21/0
orgin->sink: org pre->post, reply pre->post dev=8->5/5->8 gwy=172.27.1.1/192.168.192.65
hook=post dir=org act=snat 192.168.192.65:55811->71.170.175.108:53(172.27.1.200:55811)
hook=pre dir=reply act=dnat 71.170.175.108:53->172.27.1.200:55811(192.168.192.65:55811)
hook=post dir=reply act=noop 71.170.175.108:53->192.168.192.65:55811(0.0.0.0:0)
misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=0
serial=00002e82 tos=ff/ff app_list=2001 app=16195 url_cat=0
dd_type=0 dd_mode=0
npu_state=0x001100
npu info: flag=0x00/0x00, offload=0/0, ips_offload=0/0, epid=0/0, ipid=0/0, vlan=0x0000/0x0000
vlifid=0/0, vtag_in=0x0000/0x0000 in_npu=0/0, out_npu=0/0, fwd_en=0/0, qid=0/0
no_ofld_reason: redir-to-ips denied-by-nturbo
session info: proto=17 proto_state=01 duration=22 expire=159 timeout=0 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=4
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=log may_dirty ndr npu npd none
statistic(bytes/packets/allow_err): org=137/2/1 reply=695/2/1 tuples=3
tx speed(Bps/kbps): 6/0 rx speed(Bps/kbps): 30/0
orgin->sink: org pre->post, reply pre->post dev=8->5/5->8 gwy=172.27.1.1/192.168.192.65
hook=post dir=org act=snat 192.168.192.65:50291->71.170.175.108:53(172.27.1.200:50291)
hook=pre dir=reply act=dnat 71.170.175.108:53->172.27.1.200:50291(192.168.192.65:50291)
hook=post dir=reply act=noop 71.170.175.108:53->192.168.192.65:50291(0.0.0.0:0)
misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=0
serial=00002e2e tos=ff/ff app_list=2001 app=16195 url_cat=0
dd_type=0 dd_mode=0
npu_state=0x101100
npu info: flag=0x00/0x00, offload=0/0, ips_offload=0/0, epid=0/0, ipid=0/0, vlan=0x0000/0x0000
vlifid=0/0, vtag_in=0x0000/0x0000 in_npu=0/0, out_npu=0/0, fwd_en=0/0, qid=0/0
no_ofld_reason: offload-denied redir-to-ips denied-by-nturbo helper
5.4.1: Plain old HTTP also denied?
session info: proto=6 proto_state=01 duration=22 expire=3597 timeout=3600 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=4
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=log may_dirty ndr npu none
statistic(bytes/packets/allow_err): org=633/6/1 reply=1343/6/1 tuples=3
tx speed(Bps/kbps): 27/0 rx speed(Bps/kbps): 59/0
orgin->sink: org pre->post, reply pre->post dev=8->5/5->8 gwy=172.27.1.1/192.168.192.65
hook=post dir=org act=snat 192.168.192.65:60222->216.58.218.106:80(172.27.1.200:60222)
hook=pre dir=reply act=dnat 216.58.218.106:80->172.27.1.200:60222(192.168.192.65:60222)
hook=post dir=reply act=noop 216.58.218.106:80->192.168.192.65:60222(0.0.0.0:0)
pos/(before,after) 0/(0,0), 0/(0,0)
misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=0
serial=00002e38 tos=ff/ff app_list=2001 app=34050 url_cat=0
dd_type=0 dd_mode=0
npu_state=0x001108
npu info: flag=0x00/0x00, offload=0/0, ips_offload=0/0, epid=0/0, ipid=0/0, vlan=0x0000/0x0000
vlifid=0/0, vtag_in=0x0000/0x0000 in_npu=0/0, out_npu=0/0, fwd_en=0/0, qid=0/0
no_ofld_reason: redir-to-ips denied-by-nturbo
Update to 5.4.2:
FGT61E4Q16000530 (root) #
Firmware upgrade in progress ...
Done.
The system is going down NOW !!
Please stand by while rebooting the system.
Restarting system.
FortiGate-61E (19:30-06.08.2016)
Ver:05000009
Serial number: FGT61E4Q16000530
CPU: 1000MHz
Total RAM: 2 GB
Initializing boot device...
Initializing MAC... nplite#0
Please wait for OS to boot, or press any key to display configuration menu......
Booting OS...
Reading boot image... 2702022 bytes.
Initializing firewall...
System is starting...
Formatting shared data partition ... done!
FGT61E4Q16000530 login: admin
Password: *******
Welcome !
FGT61E4Q16000530 #
Now:
session info: proto=6 proto_state=01 duration=115 expire=3573 timeout=3600 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=4
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=log may_dirty npu none
statistic(bytes/packets/allow_err): org=2732/53/1 reply=225081/152/1 tuples=3
tx speed(Bps/kbps): 38/0 rx speed(Bps/kbps): 3142/25
orgin->sink: org pre->post, reply pre->post dev=8->5/5->8 gwy=172.27.1.1/192.168.192.65
hook=post dir=org act=snat 192.168.192.65:63479->71.170.175.104:80(172.27.1.200:63479)
hook=pre dir=reply act=dnat 71.170.175.104:80->172.27.1.200:63479(192.168.192.65:63479)
hook=post dir=reply act=noop 71.170.175.104:80->192.168.192.65:63479(0.0.0.0:0)
pos/(before,after) 0/(0,0), 0/(0,0)
misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=0
serial=000000a2 tos=ff/ff app_list=2001 app=34050 url_cat=0
dd_type=0 dd_mode=0
npu_state=0x001d08
npu info: flag=0x81/0x81, offload=8/8, ips_offload=0/0, epid=66/69, ipid=69/66, vlan=0x0000/0x0000
vlifid=69/66, vtag_in=0x0000/0x0000 in_npu=1/1, out_npu=1/1, fwd_en=0/0, qid=3/0
NPU looks OK, but ips_offload? (Apparently this display is normal; I need to go hunt down a similar display from a real NP6+CP8/9 machine.)
UTM disabled:
session info: proto=6 proto_state=01 duration=4 expire=3595 timeout=3600 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=3
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=log may_dirty npu none
statistic(bytes/packets/allow_err): org=498/4/1 reply=1348/2/1 tuples=2
tx speed(Bps/kbps): 0/0 rx speed(Bps/kbps): 0/0
orgin->sink: org pre->post, reply pre->post dev=5->3/3->5 gwy=71.170.175.104/10.182.8.65
hook=post dir=org act=snat 10.182.8.65:53646->71.170.175.104:80(71.170.175.98:53646)
hook=pre dir=reply act=dnat 71.170.175.104:80->71.170.175.98:53646(10.182.8.65:53646)
pos/(before,after) 0/(0,0), 0/(0,0)
src_mac=00:30:48:d4:15:38
misc=0 policy_id=3 auth_info=0 chk_client_info=0 vd=0
serial=00015731 tos=ff/ff app_list=0 app=0 url_cat=0
dd_type=0 dd_mode=0
npu_state=0x000c00
npu info: flag=0x81/0x81, offload=8/8, ips_offload=0/0, epid=64/66, ipid=66/64, vlan=0x0000/0x0000
vlifid=66/64, vtag_in=0x0000/0x0000 in_npu=1/1, out_npu=1/1, fwd_en=0/0, qid=2/3
OK...
session info: proto=17 proto_state=01 duration=4 expire=10 timeout=15 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=3
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=log may_dirty npu none
statistic(bytes/packets/allow_err): org=62/1/1 reply=148/1/1 tuples=2
tx speed(Bps/kbps): 15/0 rx speed(Bps/kbps): 35/0
orgin->sink: org pre->post, reply pre->post dev=5->3/3->5 gwy=71.170.175.104/10.182.8.65
hook=post dir=org act=snat 10.182.8.65:50604->71.170.175.104:53(71.170.175.98:50604)
hook=pre dir=reply act=dnat 71.170.175.104:53->71.170.175.98:50604(10.182.8.65:50604)
src_mac=00:30:48:d4:15:38
misc=0 policy_id=3 auth_info=0 chk_client_info=0 vd=0
serial=00015730 tos=ff/ff app_list=0 app=0 url_cat=0
dd_type=0 dd_mode=0
npu_state=00000000
npu info: flag=0x00/0x00, offload=0/0, ips_offload=0/0, epid=0/0, ipid=0/0, vlan=0x0000/0x0000
vlifid=0/0, vtag_in=0x0000/0x0000 in_npu=0/0, out_npu=0/0, fwd_en=0/0, qid=0/0
no_ofld_reason:
ofld_fail_reason(kernel, drv): none/not-established, none(0)/none(0)
npu_state_err=00/04
What happened to no_ofld_reason? (This is apparently a normal condition.)
Here's a rare animal. UDP DNS, helper disabled, between my two servers. The only other offloaded UDP DNS sessions I typically see are from the firewall itself. Most DNS client implementations seem to be pretty cheesy, considering the request volume these days.
session info: proto=17 proto_state=01 duration=641 expire=57 timeout=90 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=3
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/0
state=log may_dirty br npu none
statistic(bytes/packets/allow_err): org=150/2/1 reply=606/2/1 tuples=2
tx speed(Bps/kbps): 0/0 rx speed(Bps/kbps): 0/0
orgin->sink: org pre->post, reply pre->post dev=7->6/6->7 gwy=0.0.0.0/0.0.0.0
hook=pre dir=org act=noop 71.170.175.104:49832->71.170.175.108:53(0.0.0.0:0)
hook=post dir=reply act=noop 71.170.175.108:53->71.170.175.104:49832(0.0.0.0:0)
src_mac=00:25:90:a2:ad:33
misc=0 policy_id=44 auth_info=0 chk_client_info=0 vd=1
serial=006ac455 tos=ff/ff app_list=0 app=0 url_cat=0
dd_type=0 dd_mode=0
npu_state=0x000c00
npu info: flag=0x81/0x81, offload=8/8, ips_offload=0/0, epid=67/68, ipid=68/67, vlan=0x0000/0x0000
vlifid=68/67, vtag_in=0x0000/0x0000 in_npu=1/1, out_npu=1/1, fwd_en=0/0, qid=2/2
Last modified on 04/06/2017
Comments and corrections to webmaster