Notes on Fortinet's Fortigate 60E/61E Firewall Appliance




Console displays of interest


Serial number disclosure is a risk, but if a security vendor fails that test, well...

FortiGate-61E (19:30-06.08.2016)
Ver:05000009
Serial number: FGT61E4Q16000530
CPU: 1000MHz
Total RAM: 2 GB
Initializing boot device... 
Initializing MAC... nplite#0
Please wait for OS to boot, or press any key to display configuration menu......

Booting OS...
Reading boot image... 2702032 bytes.
Initializing firewall...

System is starting...


FGT61E4Q16000530 login: admin
Password: 
Welcome !

FGT61E4Q16000530 # get hardware status 

Model name: FortiGate-61E
ASIC version: SOC3
ASIC SRAM: 64M
CPU: ARMv7
Number of CPUs: 4
RAM: 1866 MB
EMMC: 3662 MB(MLC) /dev/mmcblk0
Hard disk: 122104 MB /dev/sda
USB Flash: not available
Network Card chipset: FortiASIC NP6LITE Adapter (rev.)

FGT61E4Q16000530 # get sys status 

Version: FortiGate-61E v5.4.1,build5495,160714 (GA)
Virus-DB: 1.00123(2015-12-11 13:18)
Extended DB: 1.00000(2012-10-17 15:46)
IPS-DB: 6.00741(2015-12-01 02:30)
IPS-ETDB: 0.00000(2001-01-01 00:00)
Serial-Number: FGT61E4Q16000530
IPS Malicious URL Database: 1.00001(2015-01-01 01:01)
Botnet DB: 1.00000(2012-05-28 22:51)
BIOS version: 05000009
System Part-Number: P18817-01
Log hard disk: Available
Hostname: FGT61E4Q16000530
Operation Mode: NAT
Current virtual domain: root
Max number of virtual domains: 10
Virtual domains status: 1 in NAT mode, 0 in TP mode
Virtual domain configuration: disable
FIPS-CC mode: disable
Current HA mode: standalone
Branch point: 1064
Release Version Information: GA
System time: Sat Nov 12 08:43:23 2016

FGT61E4Q16000530 # diag hardware sysinfo cpu 

Processor: ARMv7 Processor rev 1 (v7l)
processor: 0
BogoMIPS: 2007.04

processor: 1
BogoMIPS: 2007.04

processor: 2
BogoMIPS: 2007.04

processor: 3
BogoMIPS: 2007.04

Features: swp half thumb fastmult vfp edsp thumbee vfpv3 vfpv3d16 tls 
CPU implementer: 0x41
CPU architecture: 7
CPU variant: 0x4
CPU part: 0xc09
CPU revision: 1

Hardware: FSoC3_ASIC
Revision: 0000
Serial: 0000000000000000

FGT61E4Q16000530 # diag hardware sysinfo cpumem

MemTotal:        1911672 kB
MemFree:         1393712 kB
Buffers:            4568 kB
Cached:           104772 kB
SwapCached:            0 kB
Active:           116148 kB
Inactive:          79192 kB
Active(anon):      94172 kB
Inactive(anon):    25436 kB
Active(file):      21976 kB
Inactive(file):    53756 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:         86000 kB
Mapped:            35276 kB
Shmem:             33608 kB
Slab:              19708 kB
SReclaimable:       7200 kB
SUnreclaim:        12508 kB
KernelStack:         808 kB
PageTables:         6380 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      955836 kB
Committed_AS:   15610676 kB
VmallocTotal:     663552 kB
VmallocUsed:       33752 kB
VmallocChunk:     511456 kB

FGT61E4Q16000530 # diag hardware deviceinfo disk 


Disk SYSTEM(boot)    ref:       3.6GB    type: EMMC [EMMC] dev: /dev/mmcblk0
  partition ref:     247.0MB, 206.0MB free  mounted: Y  label:  dev: /dev/mmcblk0p1(boot) start: 0
  partition ref:     256.0MB, 256.0MB free  mounted: N  label:  dev: /dev/mmcblk0p2(boot) start: 0
  partition ref:   3   2.9GB,   2.9GB free  mounted: Y  label:  dev: /dev/mmcblk0p3 start: 0

Disk Internal        ref: 258 119.2GB    type: SSD [ATA LITEON CV1-8B128] dev: /dev/sda
  partition ref: 259 117.4GB, 117.2GB free  mounted: Y  label: LOGUSEDX929BA67B dev: /dev/sda1 start: 2048

Total available disks: 2
Max SSD disks: 1  Available storage disks: 1



Quirks


Note that these apply to one or more of FortiOS 5.4.1/2/3/4 on the 61E. I'm not trying to be comprehensive here.

Update to 5.6.0...



Fortigate 61E initial config


This is just my initial template, but it illustrates a couple of elements I favor operationally: Disabling the virtual-switch (to allow filtering across all ports) and setting the inspection mode to "flow" (for NP/SP offload and nTurbo).

Eliminate virtual-switch (configure for individual ports):

FGT61E4Q16000530 # config firewall policy

FGT61E4Q16000530 (policy) # delete 1

FGT61E4Q16000530 (policy) # end

FGT61E4Q16000530 # config system dhcp server

FGT61E4Q16000530 (server) # delete 1

FGT61E4Q16000530 (server) # end

FGT61E4Q16000530 # config system virtual-switch

FGT61E4Q16000530 (virtual-switch) # delete internal

FGT61E4Q16000530 (virtual-switch) # end

Set inspection mode to "flow" (to allow NP/SP offload):

FGT61E4Q16000530 # config system settings

FGT61E4Q16000530 (settings) # set inspection-mode flow

FGT61E4Q16000530 (settings) # end

Fire up VDOMs (I use two, one NAT and one transparent):

FGT61E4Q16000530 # config system global

FGT61E4Q16000530 (global) # set vdom-admin enable

FGT61E4Q16000530 (global) # end
You will be logged out for the operation to take effect
Do you want to continue? (y/n)y

exit


FGT61E4Q16000530 # config vdom

FGT61E4Q16000530 (vdom) # edit tdmz
current vf=tdmz:3

FGT61E4Q16000530 (tdmz) # config system settings

FGT61E4Q16000530 (settings) # set opmode transparent

FGT61E4Q16000530 (settings) # set inspection-mode flow

FGT61E4Q16000530 (settings) # set manageip [I use an oddball IP; some may use an actual reachable IP]

FGT61E4Q16000530 (settings) # end

Changing to TP mode

Verify a few default settings (default to highest performance; these can be configured at many levels for other purposes):

np-accel-mode = basic
cp-accel-mode = advanced



A few session displays


These samples span FortiOS 5.4.1 - 5.4.4, and several configurations.

Typical DNS sessions:

session info: proto=17 proto_state=01 duration=20 expire=159 timeout=0 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=4
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=log may_dirty ndr npu none 
statistic(bytes/packets/allow_err): org=124/1/1 reply=437/1/1 tuples=3
tx speed(Bps/kbps): 6/0 rx speed(Bps/kbps): 21/0
orgin->sink: org pre->post, reply pre->post dev=8->5/5->8 gwy=172.27.1.1/192.168.192.65
hook=post dir=org act=snat 192.168.192.65:55811->71.170.175.108:53(172.27.1.200:55811)
hook=pre dir=reply act=dnat 71.170.175.108:53->172.27.1.200:55811(192.168.192.65:55811)
hook=post dir=reply act=noop 71.170.175.108:53->192.168.192.65:55811(0.0.0.0:0)
misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=0
serial=00002e82 tos=ff/ff app_list=2001 app=16195 url_cat=0
dd_type=0 dd_mode=0
npu_state=0x001100
npu info: flag=0x00/0x00, offload=0/0, ips_offload=0/0, epid=0/0, ipid=0/0, vlan=0x0000/0x0000
vlifid=0/0, vtag_in=0x0000/0x0000 in_npu=0/0, out_npu=0/0, fwd_en=0/0, qid=0/0
no_ofld_reason:  redir-to-ips denied-by-nturbo

session info: proto=17 proto_state=01 duration=22 expire=159 timeout=0 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=4
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=log may_dirty ndr npu npd none 
statistic(bytes/packets/allow_err): org=137/2/1 reply=695/2/1 tuples=3
tx speed(Bps/kbps): 6/0 rx speed(Bps/kbps): 30/0
orgin->sink: org pre->post, reply pre->post dev=8->5/5->8 gwy=172.27.1.1/192.168.192.65
hook=post dir=org act=snat 192.168.192.65:50291->71.170.175.108:53(172.27.1.200:50291)
hook=pre dir=reply act=dnat 71.170.175.108:53->172.27.1.200:50291(192.168.192.65:50291)
hook=post dir=reply act=noop 71.170.175.108:53->192.168.192.65:50291(0.0.0.0:0)
misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=0
serial=00002e2e tos=ff/ff app_list=2001 app=16195 url_cat=0
dd_type=0 dd_mode=0
npu_state=0x101100
npu info: flag=0x00/0x00, offload=0/0, ips_offload=0/0, epid=0/0, ipid=0/0, vlan=0x0000/0x0000
vlifid=0/0, vtag_in=0x0000/0x0000 in_npu=0/0, out_npu=0/0, fwd_en=0/0, qid=0/0
no_ofld_reason:  offload-denied redir-to-ips denied-by-nturbo helper

5.4.1: Plain old HTTP also denied?

session info: proto=6 proto_state=01 duration=22 expire=3597 timeout=3600 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=4
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=log may_dirty ndr npu none 
statistic(bytes/packets/allow_err): org=633/6/1 reply=1343/6/1 tuples=3
tx speed(Bps/kbps): 27/0 rx speed(Bps/kbps): 59/0
orgin->sink: org pre->post, reply pre->post dev=8->5/5->8 gwy=172.27.1.1/192.168.192.65
hook=post dir=org act=snat 192.168.192.65:60222->216.58.218.106:80(172.27.1.200:60222)
hook=pre dir=reply act=dnat 216.58.218.106:80->172.27.1.200:60222(192.168.192.65:60222)
hook=post dir=reply act=noop 216.58.218.106:80->192.168.192.65:60222(0.0.0.0:0)
pos/(before,after) 0/(0,0), 0/(0,0)
misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=0
serial=00002e38 tos=ff/ff app_list=2001 app=34050 url_cat=0
dd_type=0 dd_mode=0
npu_state=0x001108
npu info: flag=0x00/0x00, offload=0/0, ips_offload=0/0, epid=0/0, ipid=0/0, vlan=0x0000/0x0000
vlifid=0/0, vtag_in=0x0000/0x0000 in_npu=0/0, out_npu=0/0, fwd_en=0/0, qid=0/0
no_ofld_reason:  redir-to-ips denied-by-nturbo

Update to 5.4.2:

FGT61E4Q16000530 (root) # 

Firmware upgrade in progress ...
Done.




The system is going down NOW !!



Please stand by while rebooting the system.

Restarting system.


FortiGate-61E (19:30-06.08.2016)
Ver:05000009
Serial number: FGT61E4Q16000530
CPU: 1000MHz
Total RAM: 2 GB
Initializing boot device... 
Initializing MAC... nplite#0
Please wait for OS to boot, or press any key to display configuration menu......

Booting OS...
Reading boot image... 2702022 bytes.
Initializing firewall...

System is starting...
Formatting shared data partition ... done!


FGT61E4Q16000530 login: admin
Password: *******
Welcome !

FGT61E4Q16000530 #

Now:

session info: proto=6 proto_state=01 duration=115 expire=3573 timeout=3600 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=4
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=log may_dirty npu none 
statistic(bytes/packets/allow_err): org=2732/53/1 reply=225081/152/1 tuples=3
tx speed(Bps/kbps): 38/0 rx speed(Bps/kbps): 3142/25
orgin->sink: org pre->post, reply pre->post dev=8->5/5->8 gwy=172.27.1.1/192.168.192.65
hook=post dir=org act=snat 192.168.192.65:63479->71.170.175.104:80(172.27.1.200:63479)
hook=pre dir=reply act=dnat 71.170.175.104:80->172.27.1.200:63479(192.168.192.65:63479)
hook=post dir=reply act=noop 71.170.175.104:80->192.168.192.65:63479(0.0.0.0:0)
pos/(before,after) 0/(0,0), 0/(0,0)
misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=0
serial=000000a2 tos=ff/ff app_list=2001 app=34050 url_cat=0
dd_type=0 dd_mode=0
npu_state=0x001d08
npu info: flag=0x81/0x81, offload=8/8, ips_offload=0/0, epid=66/69, ipid=69/66, vlan=0x0000/0x0000
vlifid=69/66, vtag_in=0x0000/0x0000 in_npu=1/1, out_npu=1/1, fwd_en=0/0, qid=3/0

NPU looks OK, but ips_offload? (Apparently this display is normal; I need to go hunt down a similar display from a real NP6+CP8/9 machine.)


UTM disabled:

session info: proto=6 proto_state=01 duration=4 expire=3595 timeout=3600 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=3
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=log may_dirty npu none
statistic(bytes/packets/allow_err): org=498/4/1 reply=1348/2/1 tuples=2
tx speed(Bps/kbps): 0/0 rx speed(Bps/kbps): 0/0
orgin->sink: org pre->post, reply pre->post dev=5->3/3->5 gwy=71.170.175.104/10.182.8.65
hook=post dir=org act=snat 10.182.8.65:53646->71.170.175.104:80(71.170.175.98:53646)
hook=pre dir=reply act=dnat 71.170.175.104:80->71.170.175.98:53646(10.182.8.65:53646)
pos/(before,after) 0/(0,0), 0/(0,0)
src_mac=00:30:48:d4:15:38
misc=0 policy_id=3 auth_info=0 chk_client_info=0 vd=0
serial=00015731 tos=ff/ff app_list=0 app=0 url_cat=0
dd_type=0 dd_mode=0
npu_state=0x000c00
npu info: flag=0x81/0x81, offload=8/8, ips_offload=0/0, epid=64/66, ipid=66/64, vlan=0x0000/0x0000
vlifid=66/64, vtag_in=0x0000/0x0000 in_npu=1/1, out_npu=1/1, fwd_en=0/0, qid=2/3

OK...

session info: proto=17 proto_state=01 duration=4 expire=10 timeout=15 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=3
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=log may_dirty npu none
statistic(bytes/packets/allow_err): org=62/1/1 reply=148/1/1 tuples=2
tx speed(Bps/kbps): 15/0 rx speed(Bps/kbps): 35/0
orgin->sink: org pre->post, reply pre->post dev=5->3/3->5 gwy=71.170.175.104/10.182.8.65
hook=post dir=org act=snat 10.182.8.65:50604->71.170.175.104:53(71.170.175.98:50604)
hook=pre dir=reply act=dnat 71.170.175.104:53->71.170.175.98:50604(10.182.8.65:50604)
src_mac=00:30:48:d4:15:38
misc=0 policy_id=3 auth_info=0 chk_client_info=0 vd=0
serial=00015730 tos=ff/ff app_list=0 app=0 url_cat=0
dd_type=0 dd_mode=0
npu_state=00000000
npu info: flag=0x00/0x00, offload=0/0, ips_offload=0/0, epid=0/0, ipid=0/0, vlan=0x0000/0x0000
vlifid=0/0, vtag_in=0x0000/0x0000 in_npu=0/0, out_npu=0/0, fwd_en=0/0, qid=0/0
no_ofld_reason:
ofld_fail_reason(kernel, drv): none/not-established, none(0)/none(0)
npu_state_err=00/04

What happened to no_ofld_reason? (This is apparently a normal condition.)


Here's a rare animal. UDP DNS, helper disabled, between my two servers. The only other offloaded UDP DNS sessions I typically see are from the firewall itself. Most DNS client implementations seem to be pretty cheesy, considering the request volume these days.

session info: proto=17 proto_state=01 duration=641 expire=57 timeout=90 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=3
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/0
state=log may_dirty br npu none
statistic(bytes/packets/allow_err): org=150/2/1 reply=606/2/1 tuples=2
tx speed(Bps/kbps): 0/0 rx speed(Bps/kbps): 0/0
orgin->sink: org pre->post, reply pre->post dev=7->6/6->7 gwy=0.0.0.0/0.0.0.0
hook=pre dir=org act=noop 71.170.175.104:49832->71.170.175.108:53(0.0.0.0:0)
hook=post dir=reply act=noop 71.170.175.108:53->71.170.175.104:49832(0.0.0.0:0)
src_mac=00:25:90:a2:ad:33
misc=0 policy_id=44 auth_info=0 chk_client_info=0 vd=1
serial=006ac455 tos=ff/ff app_list=0 app=0 url_cat=0
dd_type=0 dd_mode=0
npu_state=0x000c00
npu info: flag=0x81/0x81, offload=8/8, ips_offload=0/0, epid=67/68, ipid=68/67, vlan=0x0000/0x0000
vlifid=68/67, vtag_in=0x0000/0x0000 in_npu=1/1, out_npu=1/1, fwd_en=0/0, qid=2/2





Last modified on 04/06/2017
Comments and corrections to webmaster