Nc feat/integrations (#8903)

* feat: integrations backend (WIP)

* feat: migration - source table

* feat: updated migration

* feat: integration APIs - WIP

* feat: integration - crud, acl, api tests

* feat: integration - crud, acl, api tests

* feat: integration - GUI integration

* feat: private integration config

* feat: integration GUI

* feat: delete api and source creation

* feat: add hint for input fields

* fix: source creation bugs

* refactor: placeholder text correction

* refactor: include context

* feat: integration delete with transaction

* refactor: permission scope correction and move ee logic

* refactor: migration correction and improvements

* feat: confirm dialog

* refactor: review comments

* refactor: meta service changes

* feat: add oss support - WIP

* feat: add oss support

* refactor: coderabbt suggestions

* refactor: exclude config from api response

* refactor: coderabbit review comments

* refactor: rename migration names

* fix: method name correction

* fix(nc-gui): integration ui changes

* fix(nc-gui): add edit integration ui changes

* fix(nc-gui): add shared badge in integrations list

* feat(nc-gui): duplicate integration

* fix(nocodb): add copy from id integration support in create integration api

* fix(nc-gui): update useIntegration store

* fix(nc-gui): test connection btn style update

* fix(nc-gui): update new integration modal

* feat(nc-gui): add sort integration list support

* fix(nc-gui): integration table to be center aligned

* fix(nc-gui): move form item required mark to right side

* fix: remove divider

* fix(nc-gui): add input shadow

* fix(nc-gui): base name validator error message

* fix(nc-gui): add border if search connection input has some value

* fix(nc-gui): add close btn in integration modal

* chore(nc-gui): lint

* fix(nc-gui): pr review changes

* chore(nc-gui): cleanup unused code

* chore(nc-gui): lint

* fix(nc-gui): integrationsType not found issue

* fix(nc-gui): update data source table

* fix(nc-gui): populate integration name only on input value change

* fix(nc-gui): create data source form update

* fix: type correction

* fix: label correction

* fix: font corrections

* fix: remove help text

* fix: grammar in help text

* fix(nc-gui): edit source ui changes

* fix(nc-gui): base settings modal changes & datasource search feat

* fix(nc-gui): update data source table

* fix(nc-gui): move integrations outside team & settings

* fix(nc-gui): make connections table full width

* fix(nc-gui): modal height issue in small screen

* fix(nc-gui): disable editing selected connection in edit data source

* fix(nc-gui): add data sources in base settings tab

* fix(nc-gui): ant design multiple warnings issue

* fix(nc-gui): create source page scrollbar issue

* feat(nc-gui): create connection from create source page

* chore(nc-gui): lint

* fix(nc-gui): update project members tab content margin

* chore: label text change

* fix: font changes

* chore: font corrections

* chore: integration => connection

* fix(nc-gui): disable auto editing database name on changing connection name

* fix(nc-gui): table header overflow issue

* fix(nc-gui): show connection crud operation messages in toast

* feat(nc-gui): request new integration ui

* fix(nc-gui): text area height adjust issue

* fix(nc-gui): add connection from source create issue

* fix(nc-gui): show data source details in modal

* fix(nc-gui): hide private connection option

* fix(nc-gui): user should able to edit & save connection without test connection if only title updated

* fix(nc-gui): add integration page in oss

* fix(nc-gui): typo currection

* fix(nc-gui): oss create base ui changes

* misc: minor formatting changes

* misc: formatting corrections

* fix(nc-gui): overlay close btn issue

* fix(nc-gui): some review changes

* fix(nc-gui): remove link beetween connection name & database name

* fix(nc-gui): update edit base/source modal oss

* fix(nc-gui): add db type icon in select connection

* chore(nc-gui): lint

* fix: integration list - allow access based on base level role

* fix(nc-gui): load integrations on creating integration from source create issue

* fix(nc-gui): add connection count in tab

* fix: correction in soft delete logic

* fix(nc-gui): reset use ssl on panel collapse

* fix(nc-gui): reduce select input font weight

* fix(nc-gui): update connection edit access control

* fix: integration read api correction

* fix(nc-gui): some review changes

* fix(nc-gui): labels update

* fix(nc-gui): udpate text in delete modal integration -> connection

* fix: remove permission from wrong scope

* refactor: swagger description correction

* fix(nc-gui): remove connection between source name & database name

* fix(nc-gui): test connection is not needed form source name. inflection field changes

* refactor: include integration title with source

* feat: integration pagination

* fix: remove unused prop

* fix(nc-gui): update all tables tab btns tooltip

* feat: new integration request

* refactor: replace delete statement and use assigning undefined for better performance

* feat(nc-gui): sync data support in project page

* fix(nc-gui): all sync data type list

* fix(nc-gui): close sync data modal issue

* fix(nc-gui): add bg gray color on db icon of tooltip

* fix(nc-gui): make connection as required field

* fix(nc-gui): show connection name if not found and reload page

* fix(nc-gui): show connection name in ds list

* fix(nc-gui): ssl related changes

* fix: oss permission

* fix(nc-gui): active tab issue on clicking source

* feat: include source count and sources in api response

* fix(nc-gui): add getIntegration fun in useIntegrationStore

* fix(nc-gui): source list udpate issue on updating source details

* fix(nc-gui): fix external source icon alignment

* feat: include base name and source count

* fix: query correction

* fix(nc-gui): show liked sources list in delete connection modal

* fix(nc-gui): display connection usage information in list

* fix(nc-gui): add sync data types icons

* fix(nc-gui): add pagination support in connection list

* fix(nc-gui): connection pagination issue

* fix(nc-gui): connection tab count update issue

* test(nc-gui): some of test cases updated

* fix(nc-gui): some minor review changes

* fix(nc-gui): minor ui changes

* fix(nc-gui): Cannot read properties of undefined (reading 'sub_type')

* fix(nc-gui): udpate all tables btn text

* fix(nc-gui): ui changes

* fix(nc-gui): overflow issue

* fix(nc-gui): add connection icon & back btn in modal

* fix(nc-gui): some minor ui changes

* test(nc-gui): update source restriction test cases

* chore(test): remove only from test

* fix(nc-gui): update style of delete connection modal

* test(nc-gui): update acl pw test cases

* fix(test): ws collaboration role accss test fail issue

* fix(nc-gui): add connection successfully added modal

* fix(nc-gui): update connection added modal

* fix(nc-gui): trigger sync request event on upvote

* chore(nc-gui): lint

* fix(nc-gui): add learn more btn in connection successfull modal

* fix(nc-gui): add integration docs link support

* fix(nc-gui): integration table name field text truncate issue

* fix: misc corrections

* misc: button width change

* fix(nc-gui): update icons

* fix(nc-gui): update test connection btn icons

* fix(nc-gui): all tables btn gap issue

* feat(nc-gui): search option in sync data modal

* feat(nc-gui): search connection through api

* fix(nc-gui): add base and source icon in delete connection modal

* fix: update sync request event

* fix(nc-gui): rebase conflict issue

* fix: connections text length

* fix(nc-gui): enable integration/create source supported docs option

* fix(nc-gui): update advanced option header style

---------

Co-authored-by: mertmit <mertmit99@gmail.com>
Co-authored-by: Ramesh Mane <101566080+rameshmane7218@users.noreply.github.com>
Co-authored-by: Raju Udava <86527202+dstala@users.noreply.github.com>
This commit is contained in:
Pranav C
2024-08-04 15:36:20 +05:30
committed by GitHub
parent 2b26e1a5b9
commit 9de25471b8
130 changed files with 8238 additions and 1618 deletions

View File

@@ -42,6 +42,8 @@ const isReloading = ref(false)
const isDeleteBaseModalOpen = ref(false)
const toBeDeletedBase = ref<SourceType | undefined>()
const searchQuery = ref<string>('')
async function updateIfSourceOrderIsNullOrDuplicate() {
const sourceOrderSet = new Set()
let hasNullOrDuplicates = false
@@ -263,24 +265,51 @@ const isNewBaseModalOpen = computed({
},
})
const activeSource = ref<SourceType>(null)
const activeSource = ref<SourceType | null>(null)
const openedTab = ref('erd')
const isSearchResultAvailable = () => {
return (
sources.value.filter((s) => s?.alias?.toLowerCase()?.includes(searchQuery.value?.toLowerCase())).length ||
'default'.includes(searchQuery.value?.toLowerCase())
)
}
const isOpenModal = computed({
get: () => !!activeSource.value,
set: (value) => {
if (!value) {
activeSource.value = null
}
},
})
const handleClickRow = (source: SourceType, tab?: string) => {
if (tab && tab !== openedTab.value) {
openedTab.value = tab
}
activeSource.value = source
}
</script>
<template>
<div class="flex flex-col h-full" data-testid="nc-settings-datasources-tab">
<div class="px-4 py-2 flex justify-between">
<a-breadcrumb separator=">" class="w-full cursor-pointer font-weight-bold">
<a-breadcrumb-item @click="activeSource = null">
<a class="!no-underline">Data Sources</a>
</a-breadcrumb-item>
<a-breadcrumb-item v-if="activeSource">
<span class="capitalize">{{ activeSource.alias || 'Default Source' }}</span>
</a-breadcrumb-item>
</a-breadcrumb>
<div class="px-1 pt-3 mb-6 flex items-center justify-between gap-3">
<a-input
v-model:value="searchQuery"
type="text"
class="nc-search-data-source-input !max-w-90 nc-input-sm"
placeholder="Search data source"
allow-clear
>
<template #prefix>
<GeneralIcon icon="search" class="mr-2 h-4 w-4 text-gray-500" />
</template>
</a-input>
<NcButton
v-if="!isDataSourceLimitReached && !activeSource && isUIAllowed('sourceCreate')"
v-if="!isDataSourceLimitReached && isUIAllowed('sourceCreate')"
size="large"
class="z-10 !px-2"
type="primary"
@@ -292,96 +321,137 @@ const openedTab = ref('erd')
</div>
</NcButton>
</div>
<div data-testid="nc-settings-datasources" class="flex flex-row w-full nc-data-sources-view flex-grow min-h-0">
<template v-if="activeSource">
<NcTabs v-model:activeKey="openedTab" class="nc-source-tab w-full">
<a-tab-pane key="erd">
<template #tab>
<div class="tab" data-testid="nc-erd-tab">
<div>{{ $t('title.erdView') }}</div>
</div>
</template>
<div class="h-full pt-4">
<LazyDashboardSettingsErd
class="h-full overflow-auto"
:base-id="baseId"
:source-id="activeSource.id"
:show-all-columns="false"
/>
</div>
</a-tab-pane>
<a-tab-pane v-if="sources && activeSource === sources[0]" key="audit">
<template #tab>
<div class="tab" data-testid="nc-audit-tab">
<div>{{ $t('title.auditLogs') }}</div>
</div>
</template>
<div class="p-4 h-full">
<LazyDashboardSettingsBaseAudit :source-id="activeSource.id" />
</div>
</a-tab-pane>
<a-tab-pane v-if="!activeSource.is_meta && !activeSource.is_local" key="edit">
<template #tab>
<div class="tab" data-testid="nc-connection-tab">
<div>{{ $t('labels.connectionDetails') }}</div>
</div>
</template>
<div class="p-6 mt-4 h-full overflow-auto">
<LazyDashboardSettingsDataSourcesEditBase
class="w-760px pr-5"
:source-id="activeSource.id"
@source-updated="loadBases(true)"
@close="activeSource = null"
/>
</div>
</a-tab-pane>
<div
data-testid="nc-settings-datasources"
class="flex flex-row w-full nc-data-sources-view flex-grow min-h-0"
:style="{
maxHeight: isNewBaseModalOpen ? '100%' : activeSource ? 'calc(100% - 46px)' : 'calc(100% - 66px)',
}"
>
<NcModal
v-model:visible="isOpenModal"
centered
size="large"
wrap-class-name="nc-active-data-sources-view"
@keydown.esc="activeSource = null"
>
<div v-if="activeSource" class="h-full">
<div class="px-4 pt-4 pb-2 flex items-center justify-between gap-3">
<a-breadcrumb separator=">" class="flex-1 cursor-pointer font-weight-bold">
<a-breadcrumb-item @click="activeSource = null">
<a class="!no-underline">Data Sources</a>
</a-breadcrumb-item>
<a-breadcrumb-item v-if="activeSource">
<span class="capitalize">{{ activeSource.alias || 'Default Source' }}</span>
</a-breadcrumb-item>
</a-breadcrumb>
<a-tab-pane key="acl">
<template #tab>
<div class="tab" data-testid="nc-acl-tab">
<div>{{ $t('labels.uiAcl') }}</div>
</div>
</template>
<NcButton size="small" type="text" class="nc-close-btn" @click="isOpenModal = false">
<GeneralIcon icon="close" class="text-gray-600" />
</NcButton>
</div>
<div class="pt-4 h-full">
<LazyDashboardSettingsUIAcl :source-id="activeSource.id" />
</div>
</a-tab-pane>
<a-tab-pane v-if="!activeSource.is_meta && !activeSource.is_local" key="meta-sync">
<template #tab>
<div class="tab" data-testid="nc-meta-sync-tab">
<div>{{ $t('labels.metaSync') }}</div>
<NcTabs v-model:activeKey="openedTab" class="nc-source-tab w-full h-[calc(100%_-_58px)] max-h-[calc(100%_-_58px)]">
<a-tab-pane key="erd">
<template #tab>
<div class="tab" data-testid="nc-erd-tab">
<div>{{ $t('title.erdView') }}</div>
</div>
</template>
<div class="h-full p-6">
<LazyDashboardSettingsErd
class="h-full overflow-auto"
:base-id="base.id"
:source-id="activeSource.id"
:show-all-columns="false"
/>
</div>
</template>
<div class="pt-4 h-full">
<LazyDashboardSettingsMetadata :source-id="activeSource.id" @source-synced="loadBases(true)" />
</div>
</a-tab-pane>
</NcTabs>
</template>
<div v-else class="flex flex-col w-full overflow-auto mt-1">
<div
class="overflow-y-auto nc-scrollbar-md"
:style="{
maxHeight: 'calc(100vh - 200px)',
}"
>
<div class="ds-table-head">
<div class="ds-table-row">
</a-tab-pane>
<a-tab-pane v-if="sources && activeSource === sources[0]" key="audit">
<template #tab>
<div class="tab" data-testid="nc-audit-tab">
<div>{{ $t('title.auditLogs') }}</div>
</div>
</template>
<div class="p-6 h-full">
<LazyDashboardSettingsBaseAudit :source-id="activeSource.id" />
</div>
</a-tab-pane>
<a-tab-pane v-if="!activeSource.is_meta && !activeSource.is_local" key="edit">
<template #tab>
<div class="tab" data-testid="nc-connection-tab">
<div>{{ $t('labels.connectionDetails') }}</div>
</div>
</template>
<div class="h-full">
<LazyDashboardSettingsDataSourcesEditBase
:source-id="activeSource.id"
@source-updated="loadBases(true)"
@close="activeSource = null"
/>
</div>
</a-tab-pane>
<a-tab-pane key="acl">
<template #tab>
<div class="tab" data-testid="nc-acl-tab">
<div>{{ $t('labels.uiAcl') }}</div>
</div>
</template>
<div class="p-6 h-full">
<LazyDashboardSettingsUIAcl :source-id="activeSource.id" />
</div>
</a-tab-pane>
<a-tab-pane v-if="!activeSource.is_meta && !activeSource.is_local" key="meta-sync">
<template #tab>
<div class="tab" data-testid="nc-meta-sync-tab">
<div>{{ $t('labels.metaSync') }}</div>
</div>
</template>
<div class="p-6 h-full">
<LazyDashboardSettingsMetadata :source-id="activeSource.id" @source-synced="loadBases(true)" />
</div>
</a-tab-pane>
</NcTabs>
</div>
</NcModal>
<div
class="flex flex-col w-full"
:class="{
'overflow-auto': !isNewBaseModalOpen,
}"
>
<template v-if="isNewBaseModalOpen">
<DashboardSettingsDataSourcesCreateBase
v-model:open="isNewBaseModalOpen"
:connection-type="clientType"
is-modal
@source-created="loadBases(true)"
/>
</template>
<div v-else class="ds-table overflow-y-auto nc-scrollbar-thin relative max-h-full mx-1 mb-4">
<div class="ds-table-head sticky top-0 bg-white z-10">
<div class="ds-table-row !border-0">
<div class="ds-table-col ds-table-enabled cursor-pointer">{{ $t('general.visibility') }}</div>
<div class="ds-table-col ds-table-name">{{ $t('general.name') }}</div>
<div class="ds-table-col ds-table-integration-name">{{ $t('general.connection') }} {{ $t('general.name') }}</div>
<div class="ds-table-col ds-table-type">{{ $t('general.type') }}</div>
<div class="ds-table-col ds-table-actions">{{ $t('labels.actions') }}</div>
</div>
</div>
<div class="ds-table-body">
<div class="ds-table-body relative">
<Draggable :list="sources" item-key="id" handle=".ds-table-handle" @end="moveBase">
<template #header>
<div v-if="sources[0]" class="ds-table-row border-gray-200 cursor-pointer" @click="activeSource = sources[0]">
<template v-if="'default'.includes(searchQuery.toLowerCase())" #header>
<div
v-if="sources[0]"
class="ds-table-row border-gray-200 cursor-pointer"
@click="handleClickRow(sources[0], 'erd')"
>
<div class="ds-table-col ds-table-enabled">
<div class="flex items-center gap-1" @click.stop>
<div v-if="sources.length > 2" class="ds-table-handle" />
<a-tooltip>
<NcTooltip>
<template #title>
<template v-if="sources[0].enabled">{{ $t('activity.hideInUI') }}</template>
<template v-else>{{ $t('activity.showInUI') }}</template>
@@ -392,7 +462,7 @@ const openedTab = ref('erd')
size="small"
@change="toggleBase(sources[0], $event)"
/>
</a-tooltip>
</NcTooltip>
</div>
</div>
<div class="ds-table-col ds-table-name font-medium">
@@ -402,6 +472,9 @@ const openedTab = ref('erd')
</div>
</div>
<div class="ds-table-col ds-table-integration-name">
<div class="flex items-center gap-1">-</div>
</div>
<div class="ds-table-col ds-table-type">
<div class="flex items-center gap-1">-</div>
</div>
@@ -420,11 +493,18 @@ const openedTab = ref('erd')
</div>
</template>
<template #item="{ element: source, index }">
<div v-if="index !== 0" class="ds-table-row border-gray-200 cursor-pointer" @click="activeSource = source">
<div
v-if="index !== 0"
class="ds-table-row border-gray-200 cursor-pointer"
:class="{
'!hidden': !source?.alias?.toLowerCase()?.includes(searchQuery.toLowerCase()),
}"
@click="activeSource = source"
>
<div class="ds-table-col ds-table-enabled">
<div class="flex items-center gap-1" @click.stop>
<GeneralIcon v-if="sources.length > 2" icon="dragVertical" small class="ds-table-handle" />
<a-tooltip>
<NcTooltip>
<template #title>
<template v-if="source.enabled">{{ $t('activity.hideInUI') }}</template>
<template v-else>{{ $t('activity.showInUI') }}</template>
@@ -435,48 +515,91 @@ const openedTab = ref('erd')
size="small"
@change="toggleBase(source, $event)"
/>
</a-tooltip>
</NcTooltip>
</div>
</div>
<div class="ds-table-col ds-table-name font-medium w-full">
<div v-if="source.is_meta || source.is_local" class="h-8 w-1">-</div>
<span v-else class="truncate">
<NcTooltip v-else class="truncate" show-on-truncate-only>
<template #title>
{{ source.is_meta || source.is_local ? $t('general.base') : source.alias }}
</template>
{{ source.is_meta || source.is_local ? $t('general.base') : source.alias }}
</span>
</NcTooltip>
</div>
<div class="ds-table-col ds-table-integration-name font-medium w-full">
<NcTooltip class="truncate" show-on-truncate-only>
<template #title>
{{ source?.integration_title || '-' }}
</template>
{{ source?.integration_title || '-' }}
</NcTooltip>
</div>
<div class="ds-table-col ds-table-type">
<div class="flex items-center gap-2">
<GeneralBaseLogo :source-type="source.type" />
<span class="text-gray-700 capitalize">{{ source.type }}</span>
</div>
<NcBadge rounded="lg" class="flex items-center gap-2 px-2 py-1 !h-7 truncate">
<GeneralBaseLogo :source-type="source.type" class="flex-none" />
<NcTooltip placement="bottom" show-on-truncate-only class="text-sm truncate">
<template #title> {{ clientTypesMap[source.type]?.text || source.type }}</template>
{{ source.type && clientTypesMap[source.type] ? clientTypesMap[source.type]?.text : source.type }}
</NcTooltip>
</NcBadge>
</div>
<div class="ds-table-col justify-end gap-x-1 ds-table-actions">
<NcTooltip>
<template #title>
{{ $t('general.remove') }}
</template>
<NcButton
v-if="!source.is_meta && !source.is_local"
size="small"
class="nc-action-btn nc-delete-base cursor-pointer outline-0 !w-8 !px-1 !rounded-lg"
type="text"
@click.stop="openDeleteBase(source)"
>
<GeneralIcon icon="delete" class="text-red-500" />
</NcButton>
</NcTooltip>
<div class="ds-table-col justify-end gap-x-1 ds-table-actions" @click.stop>
<div class="flex justify-end">
<NcDropdown v-if="!source.is_meta && !source.is_local" placement="bottomRight">
<NcButton size="small" type="secondary">
<GeneralIcon icon="threeDotVertical" />
</NcButton>
<template #overlay>
<NcMenu>
<NcMenuItem @click="handleClickRow(source, 'edit')">
<GeneralIcon class="text-gray-800" icon="edit" />
<span>{{ $t('general.edit') }}</span>
</NcMenuItem>
<NcDivider />
<NcMenuItem class="!text-red-500 !hover:bg-red-50" @click.stop="openDeleteBase(source)">
<GeneralIcon icon="delete" />
{{ $t('general.remove') }}
</NcMenuItem>
</NcMenu>
</template>
</NcDropdown>
</div>
</div>
</div>
</template>
</Draggable>
<div
v-if="!isReloading && sources?.length && !isSearchResultAvailable()"
class="flex-none integration-table-empty flex items-center justify-center py-8 px-6"
>
<div class="px-2 py-6 text-gray-500 flex flex-col items-center gap-6 text-center">
<img
src="~assets/img/placeholder/no-search-result-found.png"
class="!w-[164px] flex-none"
alt="No search results found"
/>
{{ $t('title.noResultsMatchedYourSearch') }}
</div>
</div>
</div>
<div
v-show="isReloading"
class="flex items-center justify-center absolute left-0 top-0 w-full h-[calc(100%_-_45px)] z-10 pb-10 pointer-events-none"
>
<div class="flex flex-col justify-center items-center gap-2">
<GeneralLoader size="xlarge" />
<span class="text-center">{{ $t('general.loading') }}</span>
</div>
</div>
</div>
<LazyDashboardSettingsDataSourcesCreateBase
v-model:open="isNewBaseModalOpen"
:connection-type="clientType"
@source-created="loadBases(true)"
/>
<GeneralDeleteModal
v-model:visible="isDeleteBaseModalOpen"
:entity-name="$t('general.datasource')"
@@ -501,8 +624,11 @@ const openedTab = ref('erd')
</template>
<style scoped lang="scss">
.ds-table {
@apply border-1 border-gray-200 rounded-lg h-full;
}
.ds-table-head {
@apply flex items-center border-0 text-gray-500;
@apply flex items-center border-b-1 text-gray-500 bg-gray-50 text-sm font-weight-500;
}
.ds-table-body {
@@ -522,15 +648,19 @@ const openedTab = ref('erd')
}
.ds-table-name {
@apply col-span-9 items-center capitalize;
@apply col-span-6 items-center capitalize;
}
.ds-table-integration-name {
@apply col-span-5 items-center capitalize;
}
.ds-table-type {
@apply col-span-2 items-center;
@apply col-span-3 items-center;
}
.ds-table-actions {
@apply col-span-5 flex w-full justify-center;
@apply col-span-2 flex w-full justify-center;
}
.ds-table-col:last-child {
@@ -552,3 +682,16 @@ const openedTab = ref('erd')
@apply !min-h-0 !flex-shrink;
}
</style>
<style lang="scss">
.nc-active-data-sources-view {
.ant-modal-content {
@apply overflow-hidden;
}
.nc-modal {
@apply !p-0;
height: min(calc(100vh - 100px), 1024px);
max-height: min(calc(100vh - 100px), 1024px) !important;
}
}
</style>