Commit c3673464 authored by Karen Xie's avatar Karen Xie Committed by James Bottomley

[SCSI] cxgb3i: Add cxgb3i iSCSI driver.

This patch implements the cxgb3i iscsi connection acceleration for the
open-iscsi initiator.

The cxgb3i driver offers the iscsi PDU based offload:
- digest insertion and verification
- payload direct-placement into host memory buffer.
Signed-off-by: default avatarKaren Xie <kxie@chelsio.com>
Signed-off-by: default avatarJames Bottomley <James.Bottomley@HansenPartnership.com>
parent b632ade2
Chelsio S3 iSCSI Driver for Linux
Introduction
============
The Chelsio T3 ASIC based Adapters (S310, S320, S302, S304, Mezz cards, etc.
series of products) supports iSCSI acceleration and iSCSI Direct Data Placement
(DDP) where the hardware handles the expensive byte touching operations, such
as CRC computation and verification, and direct DMA to the final host memory
destination:
- iSCSI PDU digest generation and verification
On transmitting, Chelsio S3 h/w computes and inserts the Header and
Data digest into the PDUs.
On receiving, Chelsio S3 h/w computes and verifies the Header and
Data digest of the PDUs.
- Direct Data Placement (DDP)
S3 h/w can directly place the iSCSI Data-In or Data-Out PDU's
payload into pre-posted final destination host-memory buffers based
on the Initiator Task Tag (ITT) in Data-In or Target Task Tag (TTT)
in Data-Out PDUs.
- PDU Transmit and Recovery
On transmitting, S3 h/w accepts the complete PDU (header + data)
from the host driver, computes and inserts the digests, decomposes
the PDU into multiple TCP segments if necessary, and transmit all
the TCP segments onto the wire. It handles TCP retransmission if
needed.
On receving, S3 h/w recovers the iSCSI PDU by reassembling TCP
segments, separating the header and data, calculating and verifying
the digests, then forwards the header to the host. The payload data,
if possible, will be directly placed into the pre-posted host DDP
buffer. Otherwise, the payload data will be sent to the host too.
The cxgb3i driver interfaces with open-iscsi initiator and provides the iSCSI
acceleration through Chelsio hardware wherever applicable.
Using the cxgb3i Driver
=======================
The following steps need to be taken to accelerates the open-iscsi initiator:
1. Load the cxgb3i driver: "modprobe cxgb3i"
The cxgb3i module registers a new transport class "cxgb3i" with open-iscsi.
* in the case of recompiling the kernel, the cxgb3i selection is located at
Device Drivers
SCSI device support --->
[*] SCSI low-level drivers --->
<M> Chelsio S3xx iSCSI support
2. Create an interface file located under /etc/iscsi/ifaces/ for the new
transport class "cxgb3i".
The content of the file should be in the following format:
iface.transport_name = cxgb3i
iface.net_ifacename = <ethX>
iface.ipaddress = <iscsi ip address>
* if iface.ipaddress is specified, <iscsi ip address> needs to be either the
same as the ethX's ip address or an address on the same subnet. Make
sure the ip address is unique in the network.
3. edit /etc/iscsi/iscsid.conf
The default setting for MaxRecvDataSegmentLength (131072) is too big,
replace "node.conn[0].iscsi.MaxRecvDataSegmentLength" to be a value no
bigger than 15360 (for example 8192):
node.conn[0].iscsi.MaxRecvDataSegmentLength = 8192
* The login would fail for a normal session if MaxRecvDataSegmentLength is
too big. A error message in the format of
"cxgb3i: ERR! MaxRecvSegmentLength <X> too big. Need to be <= <Y>."
would be logged to dmesg.
4. To direct open-iscsi traffic to go through cxgb3i's accelerated path,
"-I <iface file name>" option needs to be specified with most of the
iscsiadm command. <iface file name> is the transport interface file created
in step 2.
...@@ -352,6 +352,8 @@ config ISCSI_TCP ...@@ -352,6 +352,8 @@ config ISCSI_TCP
http://open-iscsi.org http://open-iscsi.org
source "drivers/scsi/cxgb3i/Kconfig"
config SGIWD93_SCSI config SGIWD93_SCSI
tristate "SGI WD93C93 SCSI Driver" tristate "SGI WD93C93 SCSI Driver"
depends on SGI_HAS_WD93 && SCSI depends on SGI_HAS_WD93 && SCSI
......
...@@ -126,6 +126,7 @@ obj-$(CONFIG_SCSI_HPTIOP) += hptiop.o ...@@ -126,6 +126,7 @@ obj-$(CONFIG_SCSI_HPTIOP) += hptiop.o
obj-$(CONFIG_SCSI_STEX) += stex.o obj-$(CONFIG_SCSI_STEX) += stex.o
obj-$(CONFIG_SCSI_MVSAS) += mvsas.o obj-$(CONFIG_SCSI_MVSAS) += mvsas.o
obj-$(CONFIG_PS3_ROM) += ps3rom.o obj-$(CONFIG_PS3_ROM) += ps3rom.o
obj-$(CONFIG_SCSI_CXGB3_ISCSI) += libiscsi.o libiscsi_tcp.o cxgb3i/
obj-$(CONFIG_ARM) += arm/ obj-$(CONFIG_ARM) += arm/
......
EXTRA_CFLAGS += -I$(TOPDIR)/drivers/net/cxgb3
cxgb3i-y := cxgb3i_init.o cxgb3i_iscsi.o cxgb3i_pdu.o cxgb3i_offload.o
obj-$(CONFIG_SCSI_CXGB3_ISCSI) += cxgb3i_ddp.o cxgb3i.o
config SCSI_CXGB3_ISCSI
tristate "Chelsio S3xx iSCSI support"
select CHELSIO_T3
select SCSI_ISCSI_ATTRS
---help---
This driver supports iSCSI offload for the Chelsio S3 series devices.
/*
* cxgb3i.h: Chelsio S3xx iSCSI driver.
*
* Copyright (c) 2008 Chelsio Communications, Inc.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation.
*
* Written by: Karen Xie (kxie@chelsio.com)
*/
#ifndef __CXGB3I_H__
#define __CXGB3I_H__
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/errno.h>
#include <linux/types.h>
#include <linux/list.h>
#include <linux/netdevice.h>
#include <linux/scatterlist.h>
#include <scsi/libiscsi_tcp.h>
/* from cxgb3 LLD */
#include "common.h"
#include "t3_cpl.h"
#include "t3cdev.h"
#include "cxgb3_ctl_defs.h"
#include "cxgb3_offload.h"
#include "firmware_exports.h"
#include "cxgb3i_offload.h"
#include "cxgb3i_ddp.h"
#define CXGB3I_SCSI_QDEPTH_DFLT 128
#define CXGB3I_MAX_TARGET CXGB3I_MAX_CONN
#define CXGB3I_MAX_LUN 512
#define ISCSI_PDU_NONPAYLOAD_MAX \
(sizeof(struct iscsi_hdr) + ISCSI_MAX_AHS_SIZE + 2*ISCSI_DIGEST_SIZE)
struct cxgb3i_adapter;
struct cxgb3i_hba;
struct cxgb3i_endpoint;
/**
* struct cxgb3i_hba - cxgb3i iscsi structure (per port)
*
* @snic: cxgb3i adapter containing this port
* @ndev: pointer to netdev structure
* @shost: pointer to scsi host structure
*/
struct cxgb3i_hba {
struct cxgb3i_adapter *snic;
struct net_device *ndev;
struct Scsi_Host *shost;
};
/**
* struct cxgb3i_adapter - cxgb3i adapter structure (per pci)
*
* @listhead: list head to link elements
* @lock: lock for this structure
* @tdev: pointer to t3cdev used by cxgb3 driver
* @pdev: pointer to pci dev
* @hba_cnt: # of hbas (the same as # of ports)
* @hba: all the hbas on this adapter
* @tx_max_size: max. tx packet size supported
* @rx_max_size: max. rx packet size supported
* @tag_format: ddp tag format settings
*/
struct cxgb3i_adapter {
struct list_head list_head;
spinlock_t lock;
struct t3cdev *tdev;
struct pci_dev *pdev;
unsigned char hba_cnt;
struct cxgb3i_hba *hba[MAX_NPORTS];
unsigned int tx_max_size;
unsigned int rx_max_size;
struct cxgb3i_tag_format tag_format;
};
/**
* struct cxgb3i_conn - cxgb3i iscsi connection
*
* @listhead: list head to link elements
* @cep: pointer to iscsi_endpoint structure
* @conn: pointer to iscsi_conn structure
* @hba: pointer to the hba this conn. is going through
* @task_idx_bits: # of bits needed for session->cmds_max
*/
struct cxgb3i_conn {
struct list_head list_head;
struct cxgb3i_endpoint *cep;
struct iscsi_conn *conn;
struct cxgb3i_hba *hba;
unsigned int task_idx_bits;
};
/**
* struct cxgb3i_endpoint - iscsi tcp endpoint
*
* @c3cn: the h/w tcp connection representation
* @hba: pointer to the hba this conn. is going through
* @cconn: pointer to the associated cxgb3i iscsi connection
*/
struct cxgb3i_endpoint {
struct s3_conn *c3cn;
struct cxgb3i_hba *hba;
struct cxgb3i_conn *cconn;
};
int cxgb3i_iscsi_init(void);
void cxgb3i_iscsi_cleanup(void);
struct cxgb3i_adapter *cxgb3i_adapter_add(struct t3cdev *);
void cxgb3i_adapter_remove(struct t3cdev *);
int cxgb3i_adapter_ulp_init(struct cxgb3i_adapter *);
void cxgb3i_adapter_ulp_cleanup(struct cxgb3i_adapter *);
struct cxgb3i_hba *cxgb3i_hba_find_by_netdev(struct net_device *);
struct cxgb3i_hba *cxgb3i_hba_host_add(struct cxgb3i_adapter *,
struct net_device *);
void cxgb3i_hba_host_remove(struct cxgb3i_hba *);
int cxgb3i_pdu_init(void);
void cxgb3i_pdu_cleanup(void);
void cxgb3i_conn_cleanup_task(struct iscsi_task *);
int cxgb3i_conn_alloc_pdu(struct iscsi_task *, u8);
int cxgb3i_conn_init_pdu(struct iscsi_task *, unsigned int, unsigned int);
int cxgb3i_conn_xmit_pdu(struct iscsi_task *);
void cxgb3i_release_itt(struct iscsi_task *task, itt_t hdr_itt);
int cxgb3i_reserve_itt(struct iscsi_task *task, itt_t *hdr_itt);
#endif
This diff is collapsed.
/*
* cxgb3i_ddp.h: Chelsio S3xx iSCSI DDP Manager.
*
* Copyright (c) 2008 Chelsio Communications, Inc.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation.
*
* Written by: Karen Xie (kxie@chelsio.com)
*/
#ifndef __CXGB3I_ULP2_DDP_H__
#define __CXGB3I_ULP2_DDP_H__
/**
* struct cxgb3i_tag_format - cxgb3i ulp tag format for an iscsi entity
*
* @sw_bits: # of bits used by iscsi software layer
* @rsvd_bits: # of bits used by h/w
* @rsvd_shift: h/w bits shift left
* @rsvd_mask: reserved bit mask
*/
struct cxgb3i_tag_format {
unsigned char sw_bits;
unsigned char rsvd_bits;
unsigned char rsvd_shift;
unsigned char filler[1];
u32 rsvd_mask;
};
/**
* struct cxgb3i_gather_list - cxgb3i direct data placement memory
*
* @tag: ddp tag
* @length: total data buffer length
* @offset: initial offset to the 1st page
* @nelem: # of pages
* @pages: page pointers
* @phys_addr: physical address
*/
struct cxgb3i_gather_list {
u32 tag;
unsigned int length;
unsigned int offset;
unsigned int nelem;
struct page **pages;
dma_addr_t phys_addr[0];
};
/**
* struct cxgb3i_ddp_info - cxgb3i direct data placement for pdu payload
*
* @list: list head to link elements
* @tdev: pointer to t3cdev used by cxgb3 driver
* @max_txsz: max tx packet size for ddp
* @max_rxsz: max rx packet size for ddp
* @llimit: lower bound of the page pod memory
* @ulimit: upper bound of the page pod memory
* @nppods: # of page pod entries
* @idx_last: page pod entry last used
* @idx_bits: # of bits the pagepod index would take
* @idx_mask: pagepod index mask
* @rsvd_tag_mask: tag mask
* @map_lock: lock to synchonize access to the page pod map
* @gl_map: ddp memory gather list
* @gl_skb: skb used to program the pagepod
*/
struct cxgb3i_ddp_info {
struct list_head list;
struct t3cdev *tdev;
struct pci_dev *pdev;
unsigned int max_txsz;
unsigned int max_rxsz;
unsigned int llimit;
unsigned int ulimit;
unsigned int nppods;
unsigned int idx_last;
unsigned char idx_bits;
unsigned char filler[3];
u32 idx_mask;
u32 rsvd_tag_mask;
spinlock_t map_lock;
struct cxgb3i_gather_list **gl_map;
struct sk_buff **gl_skb;
};
#define ULP2_MAX_PKT_SIZE 16224
#define ULP2_MAX_PDU_PAYLOAD (ULP2_MAX_PKT_SIZE - ISCSI_PDU_NONPAYLOAD_MAX)
#define PPOD_PAGES_MAX 4
#define PPOD_PAGES_SHIFT 2 /* 4 pages per pod */
/*
* struct pagepod_hdr, pagepod - pagepod format
*/
struct pagepod_hdr {
u32 vld_tid;
u32 pgsz_tag_clr;
u32 maxoffset;
u32 pgoffset;
u64 rsvd;
};
struct pagepod {
struct pagepod_hdr hdr;
u64 addr[PPOD_PAGES_MAX + 1];
};
#define PPOD_SIZE sizeof(struct pagepod) /* 64 */
#define PPOD_SIZE_SHIFT 6
#define PPOD_COLOR_SHIFT 0
#define PPOD_COLOR_SIZE 6
#define PPOD_COLOR_MASK ((1 << PPOD_COLOR_SIZE) - 1)
#define PPOD_IDX_SHIFT PPOD_COLOR_SIZE
#define PPOD_IDX_MAX_SIZE 24
#define S_PPOD_TID 0
#define M_PPOD_TID 0xFFFFFF
#define V_PPOD_TID(x) ((x) << S_PPOD_TID)
#define S_PPOD_VALID 24
#define V_PPOD_VALID(x) ((x) << S_PPOD_VALID)
#define F_PPOD_VALID V_PPOD_VALID(1U)
#define S_PPOD_COLOR 0
#define M_PPOD_COLOR 0x3F
#define V_PPOD_COLOR(x) ((x) << S_PPOD_COLOR)
#define S_PPOD_TAG 6
#define M_PPOD_TAG 0xFFFFFF
#define V_PPOD_TAG(x) ((x) << S_PPOD_TAG)
#define S_PPOD_PGSZ 30
#define M_PPOD_PGSZ 0x3
#define V_PPOD_PGSZ(x) ((x) << S_PPOD_PGSZ)
/*
* large memory chunk allocation/release
* use vmalloc() if kmalloc() fails
*/
static inline void *cxgb3i_alloc_big_mem(unsigned int size,
gfp_t gfp)
{
void *p = kmalloc(size, gfp);
if (!p)
p = vmalloc(size);
if (p)
memset(p, 0, size);
return p;
}
static inline void cxgb3i_free_big_mem(void *addr)
{
if (is_vmalloc_addr(addr))
vfree(addr);
else
kfree(addr);
}
/*
* cxgb3i ddp tag are 32 bits, it consists of reserved bits used by h/w and
* non-reserved bits that can be used by the iscsi s/w.
* The reserved bits are identified by the rsvd_bits and rsvd_shift fields
* in struct cxgb3i_tag_format.
*
* The upper most reserved bit can be used to check if a tag is ddp tag or not:
* if the bit is 0, the tag is a valid ddp tag
*/
/**
* cxgb3i_is_ddp_tag - check if a given tag is a hw/ddp tag
* @tformat: tag format information
* @tag: tag to be checked
*
* return true if the tag is a ddp tag, false otherwise.
*/
static inline int cxgb3i_is_ddp_tag(struct cxgb3i_tag_format *tformat, u32 tag)
{
return !(tag & (1 << (tformat->rsvd_bits + tformat->rsvd_shift - 1)));
}
/**
* cxgb3i_sw_tag_usable - check if a given s/w tag has enough bits left for
* the reserved/hw bits
* @tformat: tag format information
* @sw_tag: s/w tag to be checked
*
* return true if the tag is a ddp tag, false otherwise.
*/
static inline int cxgb3i_sw_tag_usable(struct cxgb3i_tag_format *tformat,
u32 sw_tag)
{
sw_tag >>= (32 - tformat->rsvd_bits);
return !sw_tag;
}
/**
* cxgb3i_set_non_ddp_tag - mark a given s/w tag as an invalid ddp tag
* @tformat: tag format information
* @sw_tag: s/w tag to be checked
*
* insert 1 at the upper most reserved bit to mark it as an invalid ddp tag.
*/
static inline u32 cxgb3i_set_non_ddp_tag(struct cxgb3i_tag_format *tformat,
u32 sw_tag)
{
unsigned char shift = tformat->rsvd_bits + tformat->rsvd_shift - 1;
u32 mask = (1 << shift) - 1;
if (sw_tag && (sw_tag & ~mask)) {
u32 v1 = sw_tag & ((1 << shift) - 1);
u32 v2 = (sw_tag >> (shift - 1)) << shift;
return v2 | v1 | 1 << shift;
}
return sw_tag | 1 << shift;
}
/**
* cxgb3i_ddp_tag_base - shift the s/w tag bits so that reserved bits are not
* used.
* @tformat: tag format information
* @sw_tag: s/w tag to be checked
*/
static inline u32 cxgb3i_ddp_tag_base(struct cxgb3i_tag_format *tformat,
u32 sw_tag)
{
u32 mask = (1 << tformat->rsvd_shift) - 1;
if (sw_tag && (sw_tag & ~mask)) {
u32 v1 = sw_tag & mask;
u32 v2 = sw_tag >> tformat->rsvd_shift;
v2 <<= tformat->rsvd_shift + tformat->rsvd_bits;
return v2 | v1;
}
return sw_tag;
}
/**
* cxgb3i_tag_rsvd_bits - get the reserved bits used by the h/w
* @tformat: tag format information
* @tag: tag to be checked
*
* return the reserved bits in the tag
*/
static inline u32 cxgb3i_tag_rsvd_bits(struct cxgb3i_tag_format *tformat,
u32 tag)
{
if (cxgb3i_is_ddp_tag(tformat, tag))
return (tag >> tformat->rsvd_shift) & tformat->rsvd_mask;
return 0;
}
/**
* cxgb3i_tag_nonrsvd_bits - get the non-reserved bits used by the s/w
* @tformat: tag format information
* @tag: tag to be checked
*
* return the non-reserved bits in the tag.
*/
static inline u32 cxgb3i_tag_nonrsvd_bits(struct cxgb3i_tag_format *tformat,
u32 tag)
{
unsigned char shift = tformat->rsvd_bits + tformat->rsvd_shift - 1;
u32 v1, v2;
if (cxgb3i_is_ddp_tag(tformat, tag)) {
v1 = tag & ((1 << tformat->rsvd_shift) - 1);
v2 = (tag >> (shift + 1)) << tformat->rsvd_shift;
} else {
u32 mask = (1 << shift) - 1;
tag &= ~(1 << shift);
v1 = tag & mask;
v2 = (tag >> 1) & ~mask;
}
return v1 | v2;
}
int cxgb3i_ddp_tag_reserve(struct t3cdev *, unsigned int tid,
struct cxgb3i_tag_format *, u32 *tag,
struct cxgb3i_gather_list *, gfp_t gfp);
void cxgb3i_ddp_tag_release(struct t3cdev *, u32 tag);
struct cxgb3i_gather_list *cxgb3i_ddp_make_gl(unsigned int xferlen,
struct scatterlist *sgl,
unsigned int sgcnt,
struct pci_dev *pdev,
gfp_t gfp);
void cxgb3i_ddp_release_gl(struct cxgb3i_gather_list *gl,
struct pci_dev *pdev);
int cxgb3i_setup_conn_host_pagesize(struct t3cdev *, unsigned int tid,
int reply);
int cxgb3i_setup_conn_pagesize(struct t3cdev *, unsigned int tid, int reply,
unsigned long pgsz);
int cxgb3i_setup_conn_digest(struct t3cdev *, unsigned int tid,
int hcrc, int dcrc, int reply);
int cxgb3i_ddp_find_page_index(unsigned long pgsz);
int cxgb3i_adapter_ddp_init(struct t3cdev *, struct cxgb3i_tag_format *,
unsigned int *txsz, unsigned int *rxsz);
void cxgb3i_adapter_ddp_cleanup(struct t3cdev *);
#endif
/* cxgb3i_init.c: Chelsio S3xx iSCSI driver.
*
* Copyright (c) 2008 Chelsio Communications, Inc.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation.
*
* Written by: Karen Xie (kxie@chelsio.com)
*/
#include "cxgb3i.h"
#define DRV_MODULE_NAME "cxgb3i"
#define DRV_MODULE_VERSION "1.0.0"
#define DRV_MODULE_RELDATE "Jun. 1, 2008"
static char version[] =
"Chelsio S3xx iSCSI Driver " DRV_MODULE_NAME
" v" DRV_MODULE_VERSION " (" DRV_MODULE_RELDATE ")\n";
MODULE_AUTHOR("Karen Xie <kxie@chelsio.com>");
MODULE_DESCRIPTION("Chelsio S3xx iSCSI Driver");
MODULE_LICENSE("GPL");
MODULE_VERSION(DRV_MODULE_VERSION);
static void open_s3_dev(struct t3cdev *);
static void close_s3_dev(struct t3cdev *);
static cxgb3_cpl_handler_func cxgb3i_cpl_handlers[NUM_CPL_CMDS];
static struct cxgb3_client t3c_client = {
.name = "iscsi_cxgb3",
.handlers = cxgb3i_cpl_handlers,
.add = open_s3_dev,
.remove = close_s3_dev,
};
/**
* open_s3_dev - register with cxgb3 LLD
* @t3dev: cxgb3 adapter instance
*/
static void open_s3_dev(struct t3cdev *t3dev)
{
static int vers_printed;
if (!vers_printed) {
printk(KERN_INFO "%s", version);
vers_printed = 1;
}
cxgb3i_sdev_add(t3dev, &t3c_client);
cxgb3i_adapter_add(t3dev);
}
/**
* close_s3_dev - de-register with cxgb3 LLD
* @t3dev: cxgb3 adapter instance
*/
static void close_s3_dev(struct t3cdev *t3dev)
{
cxgb3i_adapter_remove(t3dev);
cxgb3i_sdev_remove(t3dev);
}
/**
* cxgb3i_init_module - module init entry point
*
* initialize any driver wide global data structures and register itself
* with the cxgb3 module
*/
static int __init cxgb3i_init_module(void)
{
int err;
err = cxgb3i_sdev_init(cxgb3i_cpl_handlers);
if (err < 0)
return err;
err = cxgb3i_iscsi_init();
if (err < 0)
return err;
err = cxgb3i_pdu_init();
if (err < 0)
return err;
cxgb3_register_client(&t3c_client);
return 0;
}
/**
* cxgb3i_exit_module - module cleanup/exit entry point
*
* go through the driver hba list and for each hba, release any resource held.
* and unregisters iscsi transport and the cxgb3 module
*/
static void __exit cxgb3i_exit_module(void)
{
cxgb3_unregister_client(&t3c_client);
cxgb3i_pdu_cleanup();
cxgb3i_iscsi_cleanup();
cxgb3i_sdev_cleanup();
}
module_init(cxgb3i_init_module);
module_exit(cxgb3i_exit_module);
This diff is collapsed.
This diff is collapsed.
/*
* cxgb3i_offload.h: Chelsio S3xx iscsi offloaded tcp connection management
*
* Copyright (C) 2003-2008 Chelsio Communications. All rights reserved.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the LICENSE file included in this
* release for licensing terms and conditions.
*
* Written by: Dimitris Michailidis (dm@chelsio.com)
* Karen Xie (kxie@chelsio.com)
*/
#ifndef _CXGB3I_OFFLOAD_H
#define _CXGB3I_OFFLOAD_H
#include <linux/skbuff.h>
#include <net/tcp.h>
#include "common.h"
#include "adapter.h"
#include "t3cdev.h"
#include "cxgb3_offload.h"
#define cxgb3i_log_error(fmt...) printk(KERN_ERR "cxgb3i: ERR! " fmt)
#define cxgb3i_log_warn(fmt...) printk(KERN_WARNING "cxgb3i: WARN! " fmt)
#define cxgb3i_log_info(fmt...) printk(KERN_INFO "cxgb3i: " fmt)
#define cxgb3i_log_debug(fmt, args...) \
printk(KERN_INFO "cxgb3i: %s - " fmt, __func__ , ## args)
/**
* struct s3_conn - an iscsi tcp connection structure
*
* @dev: net device of with connection
* @cdev: adapter t3cdev for net device
* @flags: see c3cn_flags below
* @tid: connection id assigned by the h/w
* @qset: queue set used by connection
* @mss_idx: Maximum Segment Size table index
* @l2t: ARP resolution entry for offload packets
* @wr_max: maximum in-flight writes
* @wr_avail: number of writes available
* @wr_unacked: writes since last request for completion notification
* @wr_pending_head: head of pending write queue
* @wr_pending_tail: tail of pending write queue
* @cpl_close: skb for cpl_close_req
* @cpl_abort_req: skb for cpl_abort_req
* @cpl_abort_rpl: skb for cpl_abort_rpl
* @lock: connection status lock
* @refcnt: reference count on connection
* @state: connection state
* @saddr: source ip/port address
* @daddr: destination ip/port address
* @dst_cache: reference to destination route
* @receive_queue: received PDUs
* @write_queue: un-pushed pending writes
* @retry_timer: retry timer for various operations
* @err: connection error status
* @callback_lock: lock for opaque user context
* @user_data: opaque user context
* @rcv_nxt: next receive seq. #
* @copied_seq: head of yet unread data
* @rcv_wup: rcv_nxt on last window update sent
* @snd_nxt: next sequence we send
* @snd_una: first byte we want an ack for
* @write_seq: tail+1 of data held in send buffer
*/
struct s3_conn {
struct net_device *dev;
struct t3cdev *cdev;
unsigned long flags;
int tid;
int qset;
int mss_idx;
struct l2t_entry *l2t;
int wr_max;
int wr_avail;
int wr_unacked;
struct sk_buff *wr_pending_head;
struct sk_buff *wr_pending_tail;
struct sk_buff *cpl_close;
struct sk_buff *cpl_abort_req;
struct sk_buff *cpl_abort_rpl;
spinlock_t lock;
atomic_t refcnt;
volatile unsigned int state;
struct sockaddr_in saddr;
struct sockaddr_in daddr;
struct dst_entry *dst_cache;
struct sk_buff_head receive_queue;
struct sk_buff_head write_queue;
struct timer_list retry_timer;
int err;
rwlock_t callback_lock;
void *user_data;
u32 rcv_nxt;
u32 copied_seq;
u32 rcv_wup;
u32 snd_nxt;
u32 snd_una;
u32 write_seq;
};
/*
* connection state
*/
enum conn_states {
C3CN_STATE_CONNECTING = 1,
C3CN_STATE_ESTABLISHED,
C3CN_STATE_ACTIVE_CLOSE,
C3CN_STATE_PASSIVE_CLOSE,
C3CN_STATE_CLOSE_WAIT_1,
C3CN_STATE_CLOSE_WAIT_2,
C3CN_STATE_ABORTING,
C3CN_STATE_CLOSED,
};
static inline unsigned int c3cn_is_closing(const struct s3_conn *c3cn)
{
return c3cn->state >= C3CN_STATE_ACTIVE_CLOSE;
}
static inline unsigned int c3cn_is_established(const struct s3_conn *c3cn)
{
return c3cn->state == C3CN_STATE_ESTABLISHED;
}
/*
* Connection flags -- many to track some close related events.
*/
enum c3cn_flags {
C3CN_ABORT_RPL_RCVD, /* received one ABORT_RPL_RSS message */
C3CN_ABORT_REQ_RCVD, /* received one ABORT_REQ_RSS message */
C3CN_ABORT_RPL_PENDING, /* expecting an abort reply */
C3CN_TX_DATA_SENT, /* already sent a TX_DATA WR */
C3CN_ACTIVE_CLOSE_NEEDED, /* need to be closed */
};
/**
* cxgb3i_sdev_data - Per adapter data.
* Linked off of each Ethernet device port on the adapter.
* Also available via the t3cdev structure since we have pointers to our port
* net_device's there ...
*
* @list: list head to link elements
* @cdev: t3cdev adapter
* @client: CPL client pointer
* @ports: array of adapter ports
* @sport_map_next: next index into the port map
* @sport_map: source port map
*/
struct cxgb3i_sdev_data {
struct list_head list;
struct t3cdev *cdev;
struct cxgb3_client *client;
struct adap_ports ports;
unsigned int sport_map_next;
unsigned long sport_map[0];
};
#define NDEV2CDATA(ndev) (*(struct cxgb3i_sdev_data **)&(ndev)->ec_ptr)
#define CXGB3_SDEV_DATA(cdev) NDEV2CDATA((cdev)->lldev)
void cxgb3i_sdev_cleanup(void);
int cxgb3i_sdev_init(cxgb3_cpl_handler_func *);
void cxgb3i_sdev_add(struct t3cdev *, struct cxgb3_client *);
void cxgb3i_sdev_remove(struct t3cdev *);
struct s3_conn *cxgb3i_c3cn_create(void);
int cxgb3i_c3cn_connect(struct s3_conn *, struct sockaddr_in *);
void cxgb3i_c3cn_rx_credits(struct s3_conn *, int);
int cxgb3i_c3cn_send_pdus(struct s3_conn *, struct sk_buff *);
void cxgb3i_c3cn_release(struct s3_conn *);
/**
* cxgb3_skb_cb - control block for received pdu state and ULP mode management.
*
* @flag: see C3CB_FLAG_* below
* @ulp_mode: ULP mode/submode of sk_buff
* @seq: tcp sequence number
* @ddigest: pdu data digest
* @pdulen: recovered pdu length
* @ulp_data: scratch area for ULP
*/
struct cxgb3_skb_cb {
__u8 flags;
__u8 ulp_mode;
__u32 seq;
__u32 ddigest;
__u32 pdulen;
__u8 ulp_data[16];
};
#define CXGB3_SKB_CB(skb) ((struct cxgb3_skb_cb *)&((skb)->cb[0]))
#define skb_ulp_mode(skb) (CXGB3_SKB_CB(skb)->ulp_mode)
#define skb_ulp_ddigest(skb) (CXGB3_SKB_CB(skb)->ddigest)
#define skb_ulp_pdulen(skb) (CXGB3_SKB_CB(skb)->pdulen)
#define skb_ulp_data(skb) (CXGB3_SKB_CB(skb)->ulp_data)
enum c3cb_flags {
C3CB_FLAG_NEED_HDR = 1 << 0, /* packet needs a TX_DATA_WR header */
C3CB_FLAG_NO_APPEND = 1 << 1, /* don't grow this skb */
C3CB_FLAG_COMPL = 1 << 2, /* request WR completion */
};
/**
* sge_opaque_hdr -
* Opaque version of structure the SGE stores at skb->head of TX_DATA packets
* and for which we must reserve space.
*/
struct sge_opaque_hdr {
void *dev;
dma_addr_t addr[MAX_SKB_FRAGS + 1];
};
/* for TX: a skb must have a headroom of at least TX_HEADER_LEN bytes */
#define TX_HEADER_LEN \
(sizeof(struct tx_data_wr) + sizeof(struct sge_opaque_hdr))
/*
* get and set private ip for iscsi traffic
*/
#define cxgb3i_get_private_ipv4addr(ndev) \
(((struct port_info *)(netdev_priv(ndev)))->iscsi_ipv4addr)
#define cxgb3i_set_private_ipv4addr(ndev, addr) \
(((struct port_info *)(netdev_priv(ndev)))->iscsi_ipv4addr) = addr
/* max. connections per adapter */
#define CXGB3I_MAX_CONN 16384
#endif /* _CXGB3_OFFLOAD_H */
/*
* cxgb3i_pdu.c: Chelsio S3xx iSCSI driver.
*
* Copyright (c) 2008 Chelsio Communications, Inc.
* Copyright (c) 2008 Mike Christie
* Copyright (c) 2008 Red Hat, Inc. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation.
*
* Written by: Karen Xie (kxie@chelsio.com)
*/
#include <linux/skbuff.h>
#include <linux/crypto.h>
#include <scsi/scsi_cmnd.h>
#include <scsi/scsi_host.h>
#include "cxgb3i.h"
#include "cxgb3i_pdu.h"
#ifdef __DEBUG_CXGB3I_RX__
#define cxgb3i_rx_debug cxgb3i_log_debug
#else
#define cxgb3i_rx_debug(fmt...)
#endif
#ifdef __DEBUG_CXGB3I_TX__
#define cxgb3i_tx_debug cxgb3i_log_debug
#else
#define cxgb3i_tx_debug(fmt...)
#endif
static struct page *pad_page;
/*
* pdu receive, interact with libiscsi_tcp
*/
static inline int read_pdu_skb(struct iscsi_conn *conn, struct sk_buff *skb,
unsigned int offset, int offloaded)
{
int status = 0;
int bytes_read;
bytes_read = iscsi_tcp_recv_skb(conn, skb, offset, offloaded, &status);
switch (status) {
case ISCSI_TCP_CONN_ERR:
return -EIO;
case ISCSI_TCP_SUSPENDED:
/* no transfer - just have caller flush queue */
return bytes_read;
case ISCSI_TCP_SKB_DONE:
/*
* pdus should always fit in the skb and we should get
* segment done notifcation.
*/
iscsi_conn_printk(KERN_ERR, conn, "Invalid pdu or skb.");
return -EFAULT;
case ISCSI_TCP_SEGMENT_DONE:
return bytes_read;
default:
iscsi_conn_printk(KERN_ERR, conn, "Invalid iscsi_tcp_recv_skb "
"status %d\n", status);
return -EINVAL;
}
}
static int cxgb3i_conn_read_pdu_skb(struct iscsi_conn *conn,
struct sk_buff *skb)
{
struct iscsi_tcp_conn *tcp_conn = conn->dd_data;
bool offloaded = 0;
unsigned int offset;
int rc;
cxgb3i_rx_debug("conn 0x%p, skb 0x%p, len %u, flag 0x%x.\n",
conn, skb, skb->len, skb_ulp_mode(skb));
if (!iscsi_tcp_recv_segment_is_hdr(tcp_conn)) {
iscsi_conn_failure(conn, ISCSI_ERR_PROTO);
return -EIO;
}
if (conn->hdrdgst_en && (skb_ulp_mode(skb) & ULP2_FLAG_HCRC_ERROR)) {
iscsi_conn_failure(conn, ISCSI_ERR_HDR_DGST);
return -EIO;
}
if (conn->datadgst_en && (skb_ulp_mode(skb) & ULP2_FLAG_DCRC_ERROR)) {
iscsi_conn_failure(conn, ISCSI_ERR_DATA_DGST);
return -EIO;
}
/* iscsi hdr */
rc = read_pdu_skb(conn, skb, 0, 0);
if (rc <= 0)
return rc;
if (iscsi_tcp_recv_segment_is_hdr(tcp_conn))
return 0;
offset = rc;
if (conn->hdrdgst_en)
offset += ISCSI_DIGEST_SIZE;
/* iscsi data */
if (skb_ulp_mode(skb) & ULP2_FLAG_DATA_DDPED) {
cxgb3i_rx_debug("skb 0x%p, opcode 0x%x, data %u, ddp'ed, "
"itt 0x%x.\n",
skb,
tcp_conn->in.hdr->opcode & ISCSI_OPCODE_MASK,
tcp_conn->in.datalen,
ntohl(tcp_conn->in.hdr->itt));
offloaded = 1;
} else {
cxgb3i_rx_debug("skb 0x%p, opcode 0x%x, data %u, NOT ddp'ed, "
"itt 0x%x.\n",
skb,
tcp_conn->in.hdr->opcode & ISCSI_OPCODE_MASK,
tcp_conn->in.datalen,
ntohl(tcp_conn->in.hdr->itt));
offset += sizeof(struct cpl_iscsi_hdr_norss);
}
rc = read_pdu_skb(conn, skb, offset, offloaded);
if (rc < 0)
return rc;
else
return 0;
}
/*
* pdu transmit, interact with libiscsi_tcp
*/
static inline void tx_skb_setmode(struct sk_buff *skb, int hcrc, int dcrc)
{
u8 submode = 0;
if (hcrc)
submode |= 1;
if (dcrc)
submode |= 2;
skb_ulp_mode(skb) = (ULP_MODE_ISCSI << 4) | submode;
}
void cxgb3i_conn_cleanup_task(struct iscsi_task *task)
{
struct iscsi_tcp_task *tcp_task = task->dd_data;
/* never reached the xmit task callout */
if (tcp_task->dd_data)
kfree_skb(tcp_task->dd_data);
tcp_task->dd_data = NULL;
/* MNC - Do we need a check in case this is called but
* cxgb3i_conn_alloc_pdu has never been called on the task */
cxgb3i_release_itt(task, task->hdr_itt);
iscsi_tcp_cleanup_task(task);
}
/*
* We do not support ahs yet
*/
int cxgb3i_conn_alloc_pdu(struct iscsi_task *task, u8 opcode)
{
struct iscsi_tcp_task *tcp_task = task->dd_data;
struct sk_buff *skb;
task->hdr = NULL;
/* always allocate rooms for AHS */
skb = alloc_skb(sizeof(struct iscsi_hdr) + ISCSI_MAX_AHS_SIZE +
TX_HEADER_LEN, GFP_ATOMIC);
if (!skb)
return -ENOMEM;
cxgb3i_tx_debug("task 0x%p, opcode 0x%x, skb 0x%p.\n",
task, opcode, skb);
tcp_task->dd_data = skb;
skb_reserve(skb, TX_HEADER_LEN);
task->hdr = (struct iscsi_hdr *)skb->data;
task->hdr_max = sizeof(struct iscsi_hdr);
/* data_out uses scsi_cmd's itt */
if (opcode != ISCSI_OP_SCSI_DATA_OUT)
cxgb3i_reserve_itt(task, &task->hdr->itt);
return 0;
}
int cxgb3i_conn_init_pdu(struct iscsi_task *task, unsigned int offset,
unsigned int count)
{
struct iscsi_tcp_task *tcp_task = task->dd_data;
struct sk_buff *skb = tcp_task->dd_data;
struct iscsi_conn *conn = task->conn;
struct page *pg;
unsigned int datalen = count;
int i, padlen = iscsi_padding(count);
skb_frag_t *frag;
cxgb3i_tx_debug("task 0x%p,0x%p, offset %u, count %u, skb 0x%p.\n",
task, task->sc, offset, count, skb);
skb_put(skb, task->hdr_len);
tx_skb_setmode(skb, conn->hdrdgst_en, datalen ? conn->datadgst_en : 0);
if (!count)
return 0;
if (task->sc) {
struct scatterlist *sg;
struct scsi_data_buffer *sdb;
unsigned int sgoffset = offset;
struct page *sgpg;
unsigned int sglen;
sdb = scsi_out(task->sc);
sg = sdb->table.sgl;
for_each_sg(sdb->table.sgl, sg, sdb->table.nents, i) {
cxgb3i_tx_debug("sg %d, page 0x%p, len %u offset %u\n",
i, sg_page(sg), sg->length, sg->offset);
if (sgoffset < sg->length)
break;
sgoffset -= sg->length;
}
sgpg = sg_page(sg);
sglen = sg->length - sgoffset;
do {
int j = skb_shinfo(skb)->nr_frags;
unsigned int copy;
if (!sglen) {
sg = sg_next(sg);
sgpg = sg_page(sg);
sgoffset = 0;
sglen = sg->length;
++i;
}
copy = min(sglen, datalen);
if (j && skb_can_coalesce(skb, j, sgpg,
sg->offset + sgoffset)) {
skb_shinfo(skb)->frags[j - 1].size += copy;
} else {
get_page(sgpg);
skb_fill_page_desc(skb, j, sgpg,
sg->offset + sgoffset, copy);
}
sgoffset += copy;
sglen -= copy;
datalen -= copy;
} while (datalen);
} else {
pg = virt_to_page(task->data);
while (datalen) {
i = skb_shinfo(skb)->nr_frags;
frag = &skb_shinfo(skb)->frags[i];
get_page(pg);
frag->page = pg;
frag->page_offset = 0;
frag->size = min((unsigned int)PAGE_SIZE, datalen);
skb_shinfo(skb)->nr_frags++;
datalen -= frag->size;
pg++;
}
}
if (padlen) {
i = skb_shinfo(skb)->nr_frags;
frag = &skb_shinfo(skb)->frags[i];
frag->page = pad_page;
frag->page_offset = 0;
frag->size = padlen;
skb_shinfo(skb)->nr_frags++;
}
datalen = count + padlen;
skb->data_len += datalen;
skb->truesize += datalen;
skb->len += datalen;
return 0;
}
int cxgb3i_conn_xmit_pdu(struct iscsi_task *task)
{
struct iscsi_tcp_task *tcp_task = task->dd_data;
struct sk_buff *skb = tcp_task->dd_data;
struct iscsi_tcp_conn *tcp_conn = task->conn->dd_data;
struct cxgb3i_conn *cconn = tcp_conn->dd_data;
unsigned int datalen;
int err;
if (!skb)
return 0;
datalen = skb->data_len;
tcp_task->dd_data = NULL;
err = cxgb3i_c3cn_send_pdus(cconn->cep->c3cn, skb);
cxgb3i_tx_debug("task 0x%p, skb 0x%p, len %u/%u, rv %d.\n",
task, skb, skb->len, skb->data_len, err);
if (err > 0) {
int pdulen = err;
if (task->conn->hdrdgst_en)
pdulen += ISCSI_DIGEST_SIZE;
if (datalen && task->conn->datadgst_en)
pdulen += ISCSI_DIGEST_SIZE;
task->conn->txdata_octets += pdulen;
return 0;
}
if (err < 0 && err != -EAGAIN) {
kfree_skb(skb);
cxgb3i_tx_debug("itt 0x%x, skb 0x%p, len %u/%u, xmit err %d.\n",
task->itt, skb, skb->len, skb->data_len, err);
iscsi_conn_printk(KERN_ERR, task->conn, "xmit err %d.\n", err);
iscsi_conn_failure(task->conn, ISCSI_ERR_XMIT_FAILED);
return err;
}
/* reset skb to send when we are called again */
tcp_task->dd_data = skb;
return -EAGAIN;
}
int cxgb3i_pdu_init(void)
{
pad_page = alloc_page(GFP_KERNEL);
if (!pad_page)
return -ENOMEM;
memset(page_address(pad_page), 0, PAGE_SIZE);
return 0;
}
void cxgb3i_pdu_cleanup(void)
{
if (pad_page) {
__free_page(pad_page);
pad_page = NULL;
}
}
void cxgb3i_conn_pdu_ready(struct s3_conn *c3cn)
{
struct sk_buff *skb;
unsigned int read = 0;
struct iscsi_conn *conn = c3cn->user_data;
int err = 0;
cxgb3i_rx_debug("cn 0x%p.\n", c3cn);
read_lock(&c3cn->callback_lock);
if (unlikely(!conn || conn->suspend_rx)) {
cxgb3i_rx_debug("conn 0x%p, id %d, suspend_rx %lu!\n",
conn, conn ? conn->id : 0xFF,
conn ? conn->suspend_rx : 0xFF);
read_unlock(&c3cn->callback_lock);
return;
}
skb = skb_peek(&c3cn->receive_queue);
while (!err && skb) {
__skb_unlink(skb, &c3cn->receive_queue);
read += skb_ulp_pdulen(skb);
err = cxgb3i_conn_read_pdu_skb(conn, skb);
__kfree_skb(skb);
skb = skb_peek(&c3cn->receive_queue);
}
read_unlock(&c3cn->callback_lock);
if (c3cn) {
c3cn->copied_seq += read;
cxgb3i_c3cn_rx_credits(c3cn, read);
}
conn->rxdata_octets += read;
}
void cxgb3i_conn_tx_open(struct s3_conn *c3cn)
{
struct iscsi_conn *conn = c3cn->user_data;
cxgb3i_tx_debug("cn 0x%p.\n", c3cn);
if (conn) {
cxgb3i_tx_debug("cn 0x%p, cid %d.\n", c3cn, conn->id);
scsi_queue_work(conn->session->host, &conn->xmitwork);
}
}
void cxgb3i_conn_closing(struct s3_conn *c3cn)
{
struct iscsi_conn *conn;
read_lock(&c3cn->callback_lock);
conn = c3cn->user_data;
if (conn && c3cn->state != C3CN_STATE_ESTABLISHED)
iscsi_conn_failure(conn, ISCSI_ERR_CONN_FAILED);
read_unlock(&c3cn->callback_lock);
}
/*
* cxgb3i_ulp2.h: Chelsio S3xx iSCSI driver.
*
* Copyright (c) 2008 Chelsio Communications, Inc.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation.
*
* Written by: Karen Xie (kxie@chelsio.com)
*/
#ifndef __CXGB3I_ULP2_PDU_H__
#define __CXGB3I_ULP2_PDU_H__
struct cpl_iscsi_hdr_norss {
union opcode_tid ot;
u16 pdu_len_ddp;
u16 len;
u32 seq;
u16 urg;
u8 rsvd;
u8 status;
};
struct cpl_rx_data_ddp_norss {
union opcode_tid ot;
u16 urg;
u16 len;
u32 seq;
u32 nxt_seq;
u32 ulp_crc;
u32 ddp_status;
};
#define RX_DDP_STATUS_IPP_SHIFT 27 /* invalid pagepod */
#define RX_DDP_STATUS_TID_SHIFT 26 /* tid mismatch */
#define RX_DDP_STATUS_COLOR_SHIFT 25 /* color mismatch */
#define RX_DDP_STATUS_OFFSET_SHIFT 24 /* offset mismatch */
#define RX_DDP_STATUS_ULIMIT_SHIFT 23 /* ulimit error */
#define RX_DDP_STATUS_TAG_SHIFT 22 /* tag mismatch */
#define RX_DDP_STATUS_DCRC_SHIFT 21 /* dcrc error */
#define RX_DDP_STATUS_HCRC_SHIFT 20 /* hcrc error */
#define RX_DDP_STATUS_PAD_SHIFT 19 /* pad error */
#define RX_DDP_STATUS_PPP_SHIFT 18 /* pagepod parity error */
#define RX_DDP_STATUS_LLIMIT_SHIFT 17 /* llimit error */
#define RX_DDP_STATUS_DDP_SHIFT 16 /* ddp'able */
#define RX_DDP_STATUS_PMM_SHIFT 15 /* pagepod mismatch */
#define ULP2_FLAG_DATA_READY 0x1
#define ULP2_FLAG_DATA_DDPED 0x2
#define ULP2_FLAG_HCRC_ERROR 0x10
#define ULP2_FLAG_DCRC_ERROR 0x20
#define ULP2_FLAG_PAD_ERROR 0x40
void cxgb3i_conn_closing(struct s3_conn *);
void cxgb3i_conn_pdu_ready(struct s3_conn *c3cn);
void cxgb3i_conn_tx_open(struct s3_conn *c3cn);
#endif
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment